Papers for

health monitoring developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Vision language models struggle with detailed physiological readings

Why Do VLMs Fail at Remote Physiological Measurement? Rethinking with Physiological Grounding and Numerical Alignment

Abstract: Video-based remote physiological measurement (RPM) has broad applications, while Vision-Language Models (VLMs) provide a promising opportunity to enable end-to-end physiological monitoring for future human-aware intelligent systems. However, existing RPM methods rely on task-specific models and cannot be seamlessly integrated with the multimodal understanding and interaction capabilities of VLMs. We revisit RPM with VLMs and provide empirical evidence for two major bottlenecks: generic visual semantic representations fail to preserve fine-grained physiological cues; while discrete token prediction in autoregressive generation is inherently misaligned with continuous numerical measurement. To address these challenges, we propose VITAL (Visual-physiological Token Alignment for Language Models), a visual-physiological token alignment framework for RPM and preserving general vision tasks. VITAL retains the original visual pathway for semantic understanding and introduces an independent physiological pathway to jointly model semantic and physiological information. Moreover, we propose a structured numerical alignment approach to improve the reliability of generative heart rate estimation by enhancing numerical relationship modeling. Extensive experiments show that VITAL improves cross-domain estimation performance on multiple rPPG datasets while preserving the general visual understanding capability of VLMs, demonstrating its effectiveness as a unified visual-physiological model. Code will be released.

Tue 13 OctComputer Vision and Pattern Recognition
The gist
Measuring heart rate and other body signals from videos usually needs special models, but big vision-language models (VLMs) that understand images and text together have trouble with this. The authors found two main reasons: these models miss small physiological details in pictures, and their way of predicting words doesn’t fit well with measuring ongoing numbers like heart rates. They created a new method called VITAL that keeps normal image understanding but adds a separate system to catch body signals and better handle number predictions. Tests show VITAL works better on multiple heart rate datasets while still understanding images normally.
Open → 2610.19052v1

Multimodal fusion improves breathing and pose detection accuracy

Learning Multimodal Sensor Fusion with JEPA Representations for Breathing Activity and Pose Recognition

Abstract: Predictive video representations do not by themselves specify how a multimodal system should handle missing sensors. We transfer frozen V-JEPA2 features to a set of heterogeneous sensors comprised by RGB video, depth video, and radar time series. A shared fusion model uses modality queries, fixed sensor slots, and explicit availability to classify breathing activity and pose from all seven non-empty sensor subsets. The contribution is a controlled evaluation of subset supervision, pooling capacity, and representation transfer on 39 OMuSense-23 subjects with five subject-group folds and three seeds. With matched downstream parameter counts, the final model reaches recording-level activity macro-F1 of $75.23%\pm0.60$ averaged across seven subsets. Supervising all subsets in each batch improves the matched query-pooling control by 1.923 percentage points (conditional 95% subject-bootstrap interval [0.647, 3.209]), while query pooling shows no clear advantage over matched mean pooling under balanced supervision. Radar probes support transfer beyond pooled wavelet pixels, 64-token inference approximately preserves mean activity performance, and the tested Consensus-JEPA loss adds no measured benefit against its exact control.

Sun 11 OctComputer Vision and Pattern RecognitionArtificial Intelligence
The gist
This paper looks at how to combine signals from different sensors—video, depth, and radar—to better recognize breathing and body poses. The researchers used a shared model that can handle missing sensors, training it to perform well even when some data is unavailable. They found that training on all sensor subsets together boosts accuracy, while some advanced fusion techniques didn’t show clear improvements. Their method works well across multiple people and sensor combinations.
Open → 2610.15891v1