Emg gpt predicts hand poses from muscle signals using transformers
EMG-GPT: Predictive Pretraining on Residual-Quantized EMG Tokens for Hand Pose Estimation
Machine LearningArtificial Intelligence
Summary
Estimating hand movements using muscle signals can be tricky and often needs special training. The authors introduce EMG-GPT, a computer model that learns to predict hand poses by reading simplified muscle signals over time. It uses a method called transformers to understand sequences and make future guesses about hand positions. This approach shows good results, suggesting that training only on these muscle signals can help computers understand hand movements better.
What this means in practice
- •For wearable device developers: Design wearable sensors for hand tracking that use pretrained models to improve pose estimation from muscle signals without extra labeled data.
- •For robotics engineers: Develop robot controllers that interpret muscle signals to estimate user hand movements more accurately for prosthetics or teleoperation.
Authors
Ettore Magni, Rolandos Alexandros Potamias, Stefanos Zafeiriou, Konstantinos Barmpas
Abstract
Surface electromyography (sEMG) is a low-power, cost-effective biosignal for hand-pose estimation and gesture classification. In this work, we examine whether self-supervised pretraining on sEMG can yield transferable representations for continuous hand-pose estimation. We introduce EMG-GPT, a causal transformer-based model that operates on discrete sEMG representations from a frozen residual vector quantization (RVQ) tokenizer and learns temporal dynamics through depth-autoregressive future-code prediction. The model combines within-frame integration with causal temporal modeling while preserving the geometry of the pretrained codebook. EMG-GPT shows competitive results in both Regression and Tracking tasks, supporting EMG-only pretraining as a viable approach for learning transferable sEMG representations.