Papers for

speech interface designers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Non-invasive brain recordings decode sentence meaning into text

The Semantic Bottleneck: Leveraging Semantic Representations for Non-Invasive Speech Decoding

Abstract: Non-invasive speech decoding remains constrained by the low signal-to-noise ratio of neural recordings, which makes fine-grained reconstruction of phonemes or individual words difficult. Motivated by neuroscientific evidence that high-level semantic representations are distributed across cortical regions and evolve over slower temporal scales, we hypothesize that semantic content may provide a more suitable target for non-invasive decoding than low-level acoustic or lexical features. We introduce Brain2Semantics2Text, a method that reconstructs text through an intermediate semantic embedding space. Our model maps sentence-level MEG responses into a semantic manifold and then inverts the predicted embeddings into natural language. This semantic bottleneck enables recovery of high-level meaning without word-level alignment. We describe the core principles of the approach, its implementation, and the strategies used to mitigate the challenges of learning a reliable neural-to-semantic mapping. Finally, we compare against prior non-invasive Brain2Text methods and show improved sentence-level results.

Wed 9 SeptComputation and LanguageMachine Learning

The gist

Decoding spoken words from brain signals is usually hard because these signals are noisy. The authors found that targeting the overall meaning of sentences, rather than individual sounds or words, works better for non-invasive brain recordings. They created a system that first translates brain signals into a kind of meaning representation, then turns that into text. This approach helps recover what was being thought more accurately without needing exact word-by-word matching.

Open → 2609.10296v1