Full-duplex speech models listen and speak mostly when addressed or silent
Full-Duplex Speech Models Take the Floor When Asked, Not When Needed
Computation and LanguageHuman-Computer Interaction
Summary
Speech models that can listen and talk at the same time promise always-ready assistants. The authors found these models usually speak only when directly addressed or when the speaker pauses, rather than when there’s a false fact or a danger that needs correction. Even when given the chance, the models rarely challenge wrong statements or warn about hazards. This shows these systems lack true understanding of content and decision-making abilities about when and how to intervene in conversations.
What this means in practice
- •For voice assistant developers: Improve assistant responsiveness by designing systems that better detect when to speak beyond silence or direct address.
- •For automotive system engineers: Develop in-car voice systems that recognize when to interject warnings or correct misinformation during driving conversations.
Authors
Linkai Peng, Baorian Nuchged, Kaiqi Fu, Yuyang Yao
Abstract
Full-duplex speech models listen and speak at once, promising always-on assistants. Yet they must also decide when they should speak. Human listeners speak when addressed or when the speaker stops, but also self-select to correct a false claim, supply a missing word, or warn of danger. We ask whether full-duplex models do the same. To separate the reason to speak from the opportunity, we construct context-matched English monologues in which only the trigger utterance varies within a topic, define 10 conditions from turn-allocation rules, and compress inter-word pauses to limit opportunities created by silence. Across five model families, being addressed and silence are far more reliable triggers than false facts or hazards. Frame-level text-token probabilities in Moshi and PersonaPlex are lower for false facts than for Neutral when averaged over the first 2\,s after trigger end. Pauses or permission to interrupt do not close this gap either. Given the floor, Moshi and PersonaPlex answer most direct questions, yet the proportion of non-empty false-fact replies that challenge the claim is only .14--.15, and the proportion of hazard replies that warn of danger is .04--.07. This paper thus identifies a gap in both speech initiation and response content. Closing it requires genuine content understanding and intervention decisions grounded in it.