MyoCodec provides real-time streaming encoding of muscle signals
MyoCodec: A Streaming Neural Codec for Electromyography
Artificial Intelligence
Summary
Signals from muscles contain information used in various tasks like controlling devices or decoding speech. The authors developed MyoCodec, a system that compresses these muscle signals into compact tokens that can be processed efficiently and quickly. It works in real time and performs well across tasks like typing, hand movement tracking, and speech decoding. This makes it easier to integrate muscle signal data into advanced models such as language models.
What this means in practice
- •For wearable device developers: Integrate efficient, low-latency muscle signal encoding to support real-time user interaction on wearable EMG devices.
- •For human-computer interaction teams: Use compact tokenized EMG representations to improve interfaces that decode hand poses and typing from muscle signals.
- •For assistive technology manufacturers: Enable speech decoding and synthesis from muscle signals to build communication aids for users with speech impairments.$Commercial implications: The method allows cost-effective, real-time muscle signal decoding products for assistive communication devices.
Authors
Jihwan Lee, Kleanthis Avramidis, Junhyeok Lee, Tiantian Feng, Najim Dehak, Shrikanth Narayanan
Abstract
Neural codecs encode continuous signals into compact sequences of discrete tokens, providing an interface for efficient transmission, storage, and token-based sequence modeling. This paradigm has been widely adopted in modern speech and audio frameworks; however, the biosignal domain still lacks a neural codec designed specifically for low-bitrate streaming and generalization across diverse downstream tasks. We present MyoCodec, a streaming neural codec designed for electromyography (EMG). Inspired by recent neural audio codecs, MyoCodec combines causal Transformers with residual vector quantization to encode continuous EMG signals into different levels of EMG representations spanning from continuous latent features to discrete tokens operating at 50 Hz. Trained on twelve public EMG datasets, MyoCodec achieves favorable performance in both intrinsic codec quality and representative downstream tasks, including typing (emg2qwerty), hand-pose (emg2pose), speech decoding (emg2speech), and speech-to-EMG synthesis (speech2emg). Across these tasks, MyoCodec exhibits strong performance against prior models while providing a compact and causal EMG representation. During streaming inference, it requires compute time of only 0.482 ms for each 20 ms frame, enabling real-time streaming. Also, the discrete token representation provided by MyoCodec has the potential to support integration into language-model based approaches, creating a path toward LLM-based interactive systems, where tokenized EMG representations are directly processed into such language or speech models. Code and model weights are released.