Semantic uncertainty improves timing predictions in spoken turn-taking
Using Semantic Uncertainty to Estimate Transition Relevance in Turn-taking
Computation and Language
Summary
The problem is figuring out when someone should start talking in a conversation without waiting for the other person to finish completely. The authors show that by measuring how uncertain a computer is about what word might come next in a sentence, it can better guess natural points for switching speakers. They tested this idea using real-time listener reactions and found it outperformed other computer-only methods. This means computers can better predict when to jump into a conversation like humans do.
What this means in practice
- •For voice assistant developers: Improve timing of voice assistant responses in natural speech by detecting better moments to speak.$Commercial implications: Enables more natural and responsive voice interfaces for consumers, making digital assistants feel less interruptive and more human-like.
- •For call center technology teams: Integrate semantic uncertainty measures to anticipate speaker transitions and reduce talk-over in automated customer service conversations.
Authors
Muhammad Umair, Jan P. de Ruiter
Abstract
Turn-taking is a fundamental mechanism that governs when interlocutors speak and listen. Although Spoken Dialogue Systems (SDS) exploit a range of linguistic, acoustic, and non-verbal cues, they produce ill-timed responses in unscripted interaction. A central challenge is anticipating Transition Relevance Places (TRPs), or opportunities, not obligations, for a listener to take the floor. Human listeners do not wait for turn endings; as an utterance unfolds, they use expectations about its developing meaning to anticipate TRPs and decide whether to take the floor. We examine whether these evolving expectations can be modeled through semantic uncertainty -- an LLM-derived measure of how strongly a turn so far constrains what may plausibly come next. To do so, we sample possible continuations of an ongoing turn and use changes in semantic dispersion to identify TRPs within turns. We evaluate this account on a dataset with TRP labels derived from real-time listener responses, rather than retrospective annotation. Our approach substantially outperforms prompt-based and fine-tuned text-only baselines, providing empirical support for the view that evolving semantic constraints inform perceived turn-taking opportunities in unscripted interaction.