The LOOK Pilots: Learning Opportunities through Observer for Knowledge
The Learning Opportunities through Observer for Knowledge (LOOK) Pilots are year-long research projects funded by the AI-4-AI Lab’s Observer Pilot Awards and supported by the Heyman AI Innovation Fund. The program invites researchers from machine learning, medicine, education, social sciences, and related fields to explore new opportunities in ambulatory care research.
At the center of each project is the Observer Repository, a first-of-its-kind dataset that includes:
- Video recordings of outpatient visits
- Linked electronic health record (EHR) data
- Structured patient and provider metadata
This powerful combination allows investigators to study real clinical interactions and create tools that improve patient care, clinical workflows, training, and research.
LOOK Pilot Projects
Mapping the Dynamics of Patient-Doctor Trust Using Multimodal Behavioral Analysis
Vivienne Bihe Chi¹, Lyle Ungar¹, Sharath Chandra Guntuku¹, Anish Agarwal¹, Mark Liberman², Neville Ryant²
¹University of Pennsylvania, ²Linguistic Data Consortium
Storytelling Between Older Adults and their Clinicians: Understanding A Path Towards AI Enhanced Functional Health Literacy
Shana Stites¹, Sharnita Midgett¹, Robert Scavilla², Shimrit Keddem³, Pamela Sankar³
¹University of Pennsylvania, ²FourFront, LLC, ³Penn Medicine
AutoRELATE-Observer: Validation and multimodal enhancement of AI-based assessment for patient-centered communication
Andrea Hartzler¹, Patrick Wedgeworth¹, Barbara Lam¹, Aishwarya Raj¹, Trevor Cohen¹, Janice Sabin¹, Wanda Pratt², Brian Wood¹
¹University of Washington School of Medicine, ²University of Washington
Feasibility of Audit Log Data for Estimating Face-to-Face Time Principal Investigator and Team
Nate C. Apathy, PhD¹
¹University of Maryland School of Public Health
Reimagining the Patient Encounter: AI for Detecting Social Determinants of Health in Ambulatory Care
Aviv Y. Landau¹, Jiyoun Song², Lingjie Liu¹, Kenrick Cato¹‚²
¹University of Pennsylvania, ²Children's Hospital of Philadelphia