Voice-Light enables smooth real-time voice agent turn-taking
Voice-Light: A Full-Duplex Cascaded Voice Agent with Causal Turn-Taking and Speculative Generation
SoundComputation and Language
Summary
Humans naturally talk over each other without awkward pauses or interruptions, but most voice assistants struggle with this. The authors designed Voice-Light, a speaking system that can hear and respond at the same time, guess when a person is finishing talking, and prepare answers before the turn is totally over. To avoid mistakes, the system carefully manages sounds it might cancel and controls which audio is kept for conversation memory. Their tests show how it works on real talk, and they provide the code and data for others to build upon.
What this means in practice
- •For voice assistant developers: Build voice agents that respond faster by preparing answers during overlaps and managing turn transitions more smoothly.
- •For call center automation teams: Create automated agents that handle natural, overlapping speech and acknowledgments without interrupting or losing track of conversation history.
Authors
Bertil Braun
Abstract
Natural spoken interaction requires more than streaming ASR, language generation, and speech synthesis: a system must react to overlap without canceling on every acknowledgment, prepare a response before a turn is certain, and ensure canceled audio cannot enter conversation history. We present Voice-Light, a full-duplex cascaded voice agent that combines immediate acoustic onset, a causal adapter sharing a streaming ASR encoder, reversible playback control, and private speculative response generation. Structured tool calls execute concurrently with audible bridge speech, while browser acknowledgments make rendered audio authoritative for durable history. Locked evaluation on 1,673 real-conversation silence candidates found that an earlier learned completion checkpoint preserved a 2.70% false-cutoff rate but reached only 12.53% end-of-turn recall, compared with 95.60% for a Silero timing policy. The deployed system therefore retains a hybrid controller rather than claiming a learned-policy replacement. Across three unscripted operator-run microphone sessions, 36 measured response turns had a 758 ms median from final VAD endpoint to first server audio; 21 turns were below 800 ms. These sessions are an instrumented case study, not a controlled user evaluation. We release the synthetic data, model artifacts, evaluation code and summaries, source code, and deployment configuration supporting the result.