EFormer improves hand pose tracking from muscle signals
EFormer: Temporally Aligned Local Correction for Continuous sEMG-Based Hand Pose Tracking
Machine LearningHuman-Computer Interaction
Summary
Tracking hand movements from muscle signals recorded on the skin is hard because these signals can change over time and vary between people. The authors developed EFormer, a method that improves hand pose tracking by carefully aligning and correcting these signals using past data. EFormer combines different signal sources and temporal information to reduce tracking errors compared to a baseline system. This approach helps make wearable muscle-based hand trackers more accurate and reliable.
What this means in practice
- •For wearable device developers: Enhance muscle-signal-based continuous hand pose tracking for wearable controllers or prosthetics to improve accuracy with temporal correction methods.$Commercial implications: This paper enables selling improved wearable muscle-based hand tracking devices by providing a tested way to reduce tracking errors in real-time.
- •For robotics engineers: Integrate temporally aligned sEMG signal correction to improve real-time interpretation of human hand movements for robot teleoperation.
Authors
JiaCheng Ge, SiYu Zhang
Abstract
Surface electromyography (sEMG) provides a wearable, camera-free signal for continuous hand-motion inference. Mapping muscle activity to joint kinematics remains challenging because the recorded waveforms are indirect measurements, their relationship with motion changes over time, and individual anatomy and sensor placement alter the signal distribution. This paper presents EFormer, a residual feature-correction network built on a frozen tracking backbone. EFormer combines a high-rate event branch, temporally aligned local cross-attention, two causal rotary position embedding (RoPE) temporal layers, and a bounded, dynamically gated residual. EFormer receives 16-channel sEMG sampled at 2 kHz and fuses a 64-channel tracking representation at 25 Hz with a 128-channel event representation at 200 Hz. Cross-attention uses a nominal delay of 100 ms, a 300 ms history parameter, and a 50 ms tolerance; its causal mask restricts each query to events occurring 50-400 ms earlier. The correction scale is 0.15. The evaluated continuation-training configuration contains 585,376 trainable parameters and 5,974,508 frozen parameters. On the test set, EFormer achieves an MAE of 0.1546634 rad, an RMSE of 0.24063 rad, and an R^2 of 0.74801, compared with 0.1745326 rad, 0.2715448 rad, and 0.6791103 for the official tracking baseline. EFormer reduces MAE by 11.38% relative to the baseline. The results show that temporally aligned event-feature correction can reduce continuous hand-pose tracking error.