Ppg language model improves physiological signal and clinical record alignment
PPG-LM: A Photoplethysmography-Language Model with Multi-Level Clinical Alignment
Artificial Intelligence
Summary
Photoplethysmography (PPG) is a way to measure heart and blood flow signals often recorded by wearables and hospital devices. The authors created a new model called PPG-LM that connects these signals with descriptions from health records, helping the model understand medical context better. They developed a method to generate detailed captions explaining the signals at different time levels and trained the model to match signals with these captions. This new approach helps computers better recognize health events and improve predictions about a patient’s condition.
What this means in practice
- •For hospital data teams: Integrate PPG-LM to link continuous physiological signals with patient health records for enhanced clinical event detection and risk prediction.
- •For wearable device developers: Use PPG-LM to improve interpretation of pulse signals by connecting wearable data to clinical knowledge for better health monitoring features.
Authors
Xiaoda Wang, Minxiao Wang, Maxwell A Xu, Patrick Langer, Kaiqiao Han, Defu Cao, Xiao Luo, Yuzhe Yang, Yan Liu, Xiao Hu, Yizhou Sun, Wei Wang, Carl Yang
Abstract
Photoplethysmography (PPG) is widely recorded by clinical monitors and consumer wearables, providing a scalable source of continuous physiological information. These recordings offer an opportunity for physiological assessment at scale, but realizing this potential requires models to learn from both signal-derived physiological supervision and broader clinical context captured in electronic health records (EHRs). This involves aligning information spanning local observations, care events, and entire visits with PPG representations at corresponding temporal scales. However, existing PPG foundation models primarily rely on task-specific prediction heads, while the medical knowledge of large language models does not necessarily translate into waveform understanding. To bridge this gap, we introduce PPG-LM, the first PPG-language model family to learn physiological representations from both signal-derived supervision and broader clinical context captured in EHRs. To construct clinically grounded captions, we develop an automatic captioning pipeline that generates segment-, event-, and visit-level descriptions from signal measurements and structured EHR records. We then learn from these pairs through a two-stage framework that first establishes segment-language correspondence through contrastive learning and waveform-conditioned captioning, then extends alignment to events and visits through time-aware aggregation and temporal statement matching. Pretrained on approximately 73k hours of PPG, PPG-LM supports language-based recognition, cross-modal retrieval, and segment captioning. Experiments on MC-MED, MIMIC-III, and VitalDB show improved retrieval and caption factuality over language-model baselines and gains over PPG and time-series foundation models on multiple clinical prediction tasks.