Phenomenon graph model improves label-efficient contactless breathing and heart sensing
Phenomenon-Graph JEPA: Label-Efficient Representation Learning for Contactless Cardiorespiratory Sensing
Machine LearningEmerging Technologies
Summary
Measuring breathing and heartbeats without contact often needs lots of labeled data, which is hard to get. The authors developed a new way that learns from unlabeled signals, predicting connections between parts related to the same body functions without needing tricky comparisons. They showed this method can reduce the need for labeled data when analyzing certain radar and camera signals, though its benefit depends on how the learning is arranged. This helps build better health-monitoring systems that require less detailed training.
What this means in practice
- •For wearable device engineers: Use pretraining to improve models for contactless heart and breathing monitoring with fewer labeled examples in millimeter-wave and RGB-D sensor data.
- •For health monitoring system developers: Integrate representation learning that predicts physiological signals jointly across sensor streams to enable more label-efficient training of contactless vital sign estimation.
Tested on one dataset.
Authors
Constantino Álvarez Casado, Nhi Nguyen, Mohammad Rakibur Rahman, Le Nguyen, Manuel Lage Cañellas, Sasan Sharifipour, Miguel Bordallo López
Abstract
Millimeter-wave (mmWave) radar and RGB-D cameras can record cardiac and respiratory waveforms continuously and without contact, but labeled recordings remain scarce because every label requires a supervised acquisition session. Self-supervised pretraining can exploit the unlabeled signals, yet contrastive methods depend on signal transformations and negative pairs whose validity is uncertain for cardiorespiratory data, where time warping changes breathing rate and distant windows can share the same physiological state. We present Phenomenon-Graph JEPA, a joint-embedding predictive architecture that learns from four processed one-dimensional streams without negative pairs or synthetic augmentation in its base configuration. Each stream is encoded by a temporal convolutional branch and a band-limited spectral branch. During pretraining, the model predicts stopped target embeddings along typed edges, which connect streams assigned to the same physiological phenomenon, and forward in time within a state episode. We treat this physiological typing as a testable hypothesis and compare it with wrong-edge and all-pairs prediction graphs. In the OMuSense-23 dataset, pretraining improves label-efficiency area over matched supervised training by 3.91 percentage points (95% interval 2.08 to 5.80, Holm-adjusted p = 0.006), and by 3.74 points under a second configuration evaluated on the same test participants. However, the wrong-edge and all-pairs controls do not establish a benefit from physiological typing. Optional Takens-inspired delay coordinates improve a validation comparison with learned history, whereas two wrist-only WESAD protocols do not establish a pretraining advantage. The study therefore separates the measured benefit of predictive representations from the physiological prior used to organize their training.