Spoken language models reduce silence during real time reasoning interaction
Spoken Language Models that Think Aloud
Computation and LanguageSound
Summary
Spoken language models that try to think out loud before answering can cause awkward pauses, which makes conversations feel unnatural. The authors propose a way for these models to speak short updates while still thinking, avoiding long silent gaps. Their method balances thinking and talking asynchronously, which speeds up responses without losing accuracy. This helps machines sound more responsive and natural when answering questions aloud.
What this means in practice
- •For voice assistant developers: Improve real-time responsiveness of assistants by reducing silent pauses during complex spoken reasoning tasks.
- •For call center technology teams: Enhance interactive voice response systems to sound more natural and engaging by asynchronously sharing thought progress.
Authors
Junyi Ao, Kainan Peng, Mingbo Ma, Shun Zhang, Zhenyu Tang, Xutai Ma, Xiang Li, Yinghao Li, Yuancheng Wang, Zhizheng Wu, Haizhou Li, Qing He, Xubo Liu
Abstract
While Chain-of-Thought (CoT) reasoning has improved the capability of language models, directly applying it to Spoken Language Models (SLMs) may introduce long silent intervals under the serial "think-then-speak" paradigm, disrupting real-time spoken interaction. To address this issue, we propose an asynchronous think-aloud framework for reasoning-based SLMs within the Thinker-Talker architecture. The framework maintains a primary reasoning stream for logical deduction and a lightweight think-aloud stream that generates short, task-grounded progress utterances conditioned on the user input and the evolving reasoning state. A dynamic balance strategy coordinates the two streams at runtime, triggering additional think-aloud speech to avoid silent gaps and canceling pending utterances when the final response becomes ready. Experiments on spoken reasoning and question-answering benchmarks show that our approach substantially reduces user-audible silence during reasoning while maintaining answer accuracy comparable to that of a serial "think-then-speak" baseline, demonstrating the potential of asynchronous think-aloud for responsive interaction in SLMs.