Single earbud sensor tracks most body movements accurately
One Sensor, Whole Body - 3D Body Pose from a Single Consumer Earbud IMU
Computer Vision and Pattern RecognitionHuman-Computer Interaction
Summary
Tracking how your whole body moves usually needs many sensors worn all over. This work shows that just one common earbud sensor can guess lower body poses and foot contact points pretty well. Adding extra sensors in the shoes didn’t help and sometimes made things worse because of sensor orientation problems. The authors also found that some arm movements can be partially detected from the ear sensor, but upper body posture is mostly missed. This means the reliability of the sensor matters more than the number of sensors for wearable body tracking.
What this means in practice
- •For fitness device makers: Create motion tracking features using only an earbud sensor to monitor whole-body movement and gait in everyday scenarios.$Commercial implications: This enables fitness wearable products to track body pose and foot contact without extra hardware, reducing costs and user burden.
- •For physical therapy providers: Use head-worn IMU data alone to assess lower-body mobility and gait during patient exercises without complex sensor setups.
Authors
Zhilin Guo, Boqiao Zhang, Oszkár Urbán, Josef Bengtson, Hakan Aktas, Wenzhao Li, Siyu Hong, Kyle Fogarty, Chenliang Zhou, Ali Senguel, Cengiz Oztireli
Abstract
Consumer earbuds already stream inertial motion data from the head, one of the most widely worn sensor locations on the body. We ask how much of the 3D body pose a single such head IMU can recover, and whether adding more consumer sensors actually helps. We build a multimodal capture pipeline that records four-view RGB-D video together with an AirPods head IMU and two Striv insole IMUs, synchronize the streams post-hoc, and generate pseudo-ground-truth with SAM 3D Body, yielding a 35-take single-subject benchmark spanning gait, turning, vertical, everyday, and clinically inspired motions. Adapting two recurrent model families (IMUPoser and MobilePoser), we show that one head IMU recovers lower-body pose at 79.0 mm rigid-MPJPE and per-foot ground contact at 0.809 macro-F1, and that a causal variant retains most of this accuracy at streaming latency. In paired per-take significance tests across both families, adding the consumer foot IMUs never significantly improves pose and significantly degrades it in two of four model-split combinations; a mounting-bias probe and feet-only ablation identify insole orientation quality, not foot placement, as the mechanism. Extending the output to a 20-joint full-body skeleton maps the boundary: gross distal-arm motion is partially recoverable from the head alone, proximal upper-body pose is not, and staged fine-tuning recovers the leg accuracy that naive joint training sacrifices to multi-task dilution. For learned pose from consumer wearables, sensor reliability, not sensor count, is the binding constraint here. For the devices tested, the earbud is its sweet spot. Code is available at https://github.com/ZhilinGuo/one-sensor-whole-body.