Synthetic IMU data improves human activity recognition across subjects
Bio-PM-Guided Synthetic IMU Generation
Machine Learning
Summary
It is hard and costly to collect real motion sensor data to recognize what activities people are doing, and models often do poorly when tested on new people. The authors created a way to generate realistic synthetic sensor data by working with a special hidden representation of the original signals, not the raw signals themselves. They showed that adding this synthetic data helps activity recognition systems work better across many different people, especially for activities that are usually confusing. This approach could make it easier and cheaper to build better activity recognition tools.
What this means in practice
- •For wearable device developers: Improve activity recognition accuracy across users by augmenting training data with high-quality synthetic IMU signals.
- •For health monitoring platform teams: Enhance robustness of human activity classifiers against diverse user data by integrating synthetic data generated from pretrained biosignal representations.
Authors
Deepali Singh, Deepesh Parpani, Aviral Gupta
Abstract
Collecting and labeling inertial measurement unit (IMU) data for human activity recognition (HAR) is expensive, and models trained on one group of subjects often fail to generalize to unseen individuals. Even with balanced data, classifiers struggle to distinguish dynamically similar activity classes and benefit from targeted augmentation. We explore whether synthetic IMU data generated in the representation space of a pretrained biosignal encoder, Bio-PM \cite{b1}, can serve as a practical augmentation source. Rather than synthesizing raw signals, we leverage Bio-PM's frozen encoder to convert 10-second IMU windows into token embedding sequences and train class-conditional decoders in this space: (1) an autoregressive transformer decoder that predicts tokens sequentially, and (2) a diffusion decoder that learns the token distribution through a denoising objective. We analyze generation realism through distributional metrics and embedding-space visualizations, evaluate downstream utility through leave-one-subject-out HAR classification, and present a PatchDecoder for approximate raw waveform reconstruction. Our diffusion decoder achieves a Train-on-Synthetic-Test-on-Real (TSTR) macro-F1 of 0.68, demonstrating that generated tokens are realistic enough to train a classifier that transfers to real data. Combining synthetic-token augmentation with a 320-dimensional gravity-aware feature representation raises LOSO macro-F1 from 0.729 to 0.809 over a real-only baseline across 51 subjects on the WISDM benchmark, with statistically significant gains concentrated in the most challenging activity classes. These results demonstrate that high-fidelity generation in a pretrained encoder's representation space is achievable and practically useful for cross-subject HAR augmentation.