Garment-mounted virtual imu data created from text improves body motion sensing
IMUdrape: Generating Garment-Mounted Virtual IMU Data from Text through Physics-Based Cloth Simulation
Computer Vision and Pattern RecognitionHuman-Computer Interaction
Summary
Sensors in clothes don’t just move with the body but also with the fabric’s loose motions, making the sensor data harder to interpret. The authors developed IMUdrape, a system that uses physics-based simulations of cloth and text descriptions of activities to create virtual sensor data as if from real sensors sewn into clothes. This approach helps train models that better understand the actual motion recorded by these sensors without needing lots of real-world training data. Tests show that IMUdrape improves activity recognition and pose estimation compared to previous methods that assumed sensors are fixed to the body.
What this means in practice
- •For wearable device designers: Generate realistic training data for garment-mounted IMU sensors to improve activity and pose recognition without extensive real-world collection.
- •For sports technology developers: Enhance sensorized sportswear by simulating sensor motions in different garment designs to optimize sensor placement and data interpretation.
Authors
Zikang Leng, Thomas Plötz
Abstract
Inertial sensors are moving into clothing, from sensorized sportswear to garments instrumented with hundreds of IMUs. Sewn into clothing, a sensor no longer moves rigidly with the body: loose fabric slides, twists and swings, so a garment-mounted sensor records the motion of the fabric as well as that of the body, and this extra motion differs from one garment to the next. Models for such sensors need training data recorded with the deployed garment, yet labeled data are scarce. Virtual IMU data generated from videos or text have eased this scarcity for body-attached sensors, but existing generators attach their virtual sensors rigidly to the body. We present IMUdrape, which generates garment-mounted virtual IMU data from text through physics-based cloth simulation. IMUdrape synthesizes motion from activity descriptions, reconstructs the target garment as a sewing pattern from photographs, places virtual IMUs at the deployment's real sensor positions, and simulates the garment with an intersection-free contact solver, so that every simulation yields paired garment and body signals. On four real datasets with clothing-mounted sensors, IMUdrape improves on existing virtual-data approaches for both activity recognition and full-body pose estimation. With one real subject, its data improve recognition macro-F1 by 7% to 46% over real data alone and by 2% to 11% over body-mounted virtual data from the same motion. When the garment is simulated on recorded motion, the pose error without any real training data falls by 30% to 33% relative to generated motion, and garment data remain 12% more accurate than body data. IMUdrape thus reduces the amount of real data that garment-mounted sensing requires.