Papers for

speech interface designers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Full duplex speech models take turns later than humans in conversation

Coupled but Late: Turn-Taking Between Full-Duplex Speech Models in Unscripted Dialogue

Abstract: Full-duplex speech models are trained to converse with a person, but they are increasingly made to converse with each other, in self-play data generation, agent societies, and model-based evaluation. In that loop no human absorbs a timing error: each model's turn-taking is the other's input. We ask what timing the loop settles into. Two PersonaPlex-7B instances exchange audio tokens on a shared clock in unscripted conversation, and one floor-transfer rule is applied to them and to Switchboard. Their timing is coupled: re-pairing speakers across conversations destroys it. But the floor changes hands late, at a median of 400-560 ms against 137 ms for humans, and the last 120 ms of the partner's turn, where human projection places a tenth of its transfers, holds 1% of theirs. Delaying one direction of the channel shifts the response one-for-one and leaves the run-up to it empty, consistent with a reactive wait after the perceived end rather than the turn-end projection human timing requires.

Tue 6 OctArtificial Intelligence
The gist
When two AI speech models talk to each other, the timing of their turn-taking is linked but different from how humans do it. The researchers found that these models wait longer to switch turns, usually around 400-560 milliseconds, compared to humans who switch in about 137 milliseconds. The models seem to react after the other finishes speaking rather than predicting when the turn will end, unlike humans. This study shows that AI conversations have a different rhythm and timing than natural human conversations.
Open → 2610.08683v1

Non-invasive brain recordings decode sentence meaning into text

The Semantic Bottleneck: Leveraging Semantic Representations for Non-Invasive Speech Decoding

Abstract: Non-invasive speech decoding remains constrained by the low signal-to-noise ratio of neural recordings, which makes fine-grained reconstruction of phonemes or individual words difficult. Motivated by neuroscientific evidence that high-level semantic representations are distributed across cortical regions and evolve over slower temporal scales, we hypothesize that semantic content may provide a more suitable target for non-invasive decoding than low-level acoustic or lexical features. We introduce Brain2Semantics2Text, a method that reconstructs text through an intermediate semantic embedding space. Our model maps sentence-level MEG responses into a semantic manifold and then inverts the predicted embeddings into natural language. This semantic bottleneck enables recovery of high-level meaning without word-level alignment. We describe the core principles of the approach, its implementation, and the strategies used to mitigate the challenges of learning a reliable neural-to-semantic mapping. Finally, we compare against prior non-invasive Brain2Text methods and show improved sentence-level results.

Wed 9 SeptComputation and LanguageMachine Learning
The gist
Decoding spoken words from brain signals is usually hard because these signals are noisy. The authors found that targeting the overall meaning of sentences, rather than individual sounds or words, works better for non-invasive brain recordings. They created a system that first translates brain signals into a kind of meaning representation, then turns that into text. This approach helps recover what was being thought more accurately without needing exact word-by-word matching.
Open → 2609.10296v1