Brain signals decoded to restore speech and study language
Brain-to-Language Decoding: Tasks, Signals, Methods, Evaluation, Practical Use and Beyond
Computation and LanguageHuman-Computer InteractionNeural and Evolutionary Computing
Summary
People who lose the ability to speak could communicate again by directly translating brain activity into words or speech. The authors review progress in reading brain signals from inside and outside the skull to understand and recreate spoken language. They explain how these signals connect to what people say out loud, think silently, or hear, and how different models turn those signals into text, speech, or animated faces. The paper also covers how these systems are tested, how well they perform, and future steps toward richer communication between brains and machines.
What this means in practice
- •For assistive technology developers: Develop communication aids that convert brain signals into personalized speech for people who cannot speak.$Commercial implications: Enables creating and selling brain-based speech devices restoring communication for speech-disabled users.
- •For neurotechnology engineers: Design systems that decode various types of brain language signals to enable real-time control and feedback in human-computer interaction.
A survey. It maps existing work.
Authors
Yiqian Yang, Yiqun Duan, Chenyu Liu, Yiqi Wang, Xinliang Zhou, Chin-Teng Lin, Yu Zhang
Abstract
Brain-to-language decoding translates neural activity associated with language production, internal speech and perception into linguistic or expressive outputs. It offers a route to restoring communication after speech loss and a means of studying how the brain represents language. Advances in neural recording and representation learning have expanded the field from constrained recognition and acoustic reconstruction to text generation, streaming personalised speech and facial animation. This survey synthesises these developments across invasive and non-invasive measurements, drawing on a search without a lower year limit and source-led updates through September 2026. We connect Articulated, Inner and Perceived tasks to the neural populations they engage, the representations available to decoders and the outputs those representations can support. We examine model development, public resources and the evolution of evaluation, and compare published performance and communication costs within their reported protocols. The synthesis identifies complementary routes to progress: phonetic, acoustic and semantic targets preserve different aspects of a message; shared representations support reuse across recording conditions and tasks; and online communication increasingly depends on calibration, feedback and user control alongside decoding accuracy. Shared benchmarks enable algorithmic comparisons, while longitudinal studies reveal the demands of sustained use. We discuss these developments and their remaining limitations, then outline a prospective five-level trajectory from commands and language to meaning, scenarios and bidirectional cognitive exchange