Papers for

speech platform engineers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

VoxTurn improves voice agent turn-taking decisions with one model

VoxTurn: Answering Several Turn-Taking Questions in One Pass with One Model

Abstract: A cascaded voice agent must decide, throughout each silence of its user, whether the user has finished their turn: answering too early cuts the user off, answering too late makes the agent slow. It often needs related decisions too: will the user soon go on, is speech during its own turn an interruption or a backchannel, is a sound speech at all. With open detectors, each extra decision costs another model or another call, and on the 14-language eot-bench they cut users off in more than a quarter of mid-turn pauses. Because they are trained on segmented utterances, this mismatch is the largest single source of error. We present VoxTurn, a 0.8B speech LLM that casts these decisions as multiple-choice questions, scores the hidden state of each option, and answers any set of them in one forward pass with a block-isolating attention mask, exactly as if asked one at a time. Trained on real conversational pauses with labels derived from their timing, and re-queried while a silence lasts, VoxTurn reaches 11.5% false cut-offs in English at a 300 ms latency budget, comparable to two commercial detectors, and a 14-language mean of 19.9%, the lowest among open models, at 8.3 ms per decision. In Pipecat and LiveKit it cuts off 17% and 9% of turns, and it ranks fifth of 24 in TurnBench end-of-turn detection without any TurnBench training data. The model, code and framework plugins are released.

Thu 15 OctSound
The gist
Voice assistants must decide when users have finished speaking to respond quickly without interruptions. Current methods use multiple models for different related decisions, causing mistakes and delays. The authors present VoxTurn, a single large speech model that answers several turn-taking questions all at once. It performs well across many languages and is faster and more accurate than open alternatives, cutting user interruptions by a notable margin.
Open → 2610.21150v1