Around-ear silent speech device works well across days without recalibration
AESSI: An Around-Ear Silent Speech Interface for Cross-Day Online Reuse without Test-Day Calibration
Human-Computer Interaction
Summary
Silent speech devices let people communicate without making sounds, which is helpful for those who have trouble speaking after a stroke. The authors built a device worn around the ear that understands silent speech and works accurately even when used on different days without needing recalibration. They trained the device to handle changes like sensor movement by predicting missing parts of sound signals. The system achieved over 90% accuracy without adjustment and nearly 98% accuracy in repeated tests days later, showing it can be reused day-to-day easily.
What this means in practice
- •For brain-computer interface developers: Create silent speech communication tools that maintain accuracy across multiple days without recalibration efforts.
- •For speech therapy device manufacturers: Design wearable silent speech aids for people with speech impairments that do not require daily sensor adjustments.$Commercial implications: Enables development of reliable, easy-to-use commercial silent speech assistive devices for patients with dysarthria or other speech deficits.
Authors
Xiran Xu, Mochu Dong, Yujie Yan, Chenxi Wang, Yu Jiao, Jing Chen
Abstract
Silent speech interfaces (SSIs) enable private communication without audible speech and may support people with post-stroke dysarthria. Everyday reuse requires articulation-related representations that generalize across days despite sensor repositioning and physiological changes. We present AESSI, an around-ear SSI using masked-context representation pretraining (MCRP): a student predicts teacher representations from masked time-frequency inputs to encourage robustness to recording variability. We collected 44 electrophysiological recordings from 24 participants for 25 everyday Mandarin sentences. With six participants' complete final recordings held out, AESSI achieved 92.24 percent mean accuracy without test-day calibration. AESSI exceeded the best adapted baseline by 50.57 percentage points. At least 21 days after each participant's last recording, five participants each completed 50 independently randomized online tests without calibration, achieving 98.0 percent overall accuracy. Median preprocessing and inference time in CPU replay was 31.70 ms. These results demonstrate end-to-end system operation and support online reuse without test-day calibration. Demo is included with the paper.