Full-duplex speech models maintain quality during long calls
Diagnosing and Mitigating Long-Session Degradation in Full-Duplex Speech Models
Computation and Language
Summary
Spoken dialogue models that can talk and listen at the same time sometimes get worse when conversations last a long time. The authors studied how these models lose quality over extended use and found that they respond less well and sound less consistent. They created new ways to train and organize these models so they stay clear and responsive even in long conversations. Their improvements keep the system efficient and prevent slowdowns as the conversation grows longer.
What this means in practice
- •For voice assistant developers: Improve responsiveness and voice quality of assistants in long interactions using efficient training and memory methods.
- •For customer support platform teams: Maintain natural and consistent dialogue quality in long customer calls using lightweight model adaptations and compressed interaction history.
Authors
Junho Kim, CheHyun Lee, Donghun Kang, Youngho Cho, Dongwook Lee, Heeseung Kim, Sungroh Yoon
Abstract
End-to-end full-duplex spoken dialogue models enable natural, real-time spoken interactions with interruptions, backchannels, and overlapping speech. Prior evaluations have assessed diverse interaction capabilities and identified some degradation patterns across conversational turns. However, how model behavior evolves over extended sessions remains insufficiently understood. In this work, we address this gap by systematically characterizing long-session degradation and developing efficient methods to mitigate it. We introduce a closed-loop evaluation framework for long-session stability and apply it to several publicly available full-duplex speech models, revealing degradation over time in responsiveness, spoken content quality, and voice consistency. To mitigate this degradation, we propose a self-distillation-based training framework for lightweight LoRA adaptation that improves long-session stability with a small amount of additional training. We further introduce a two-resolution context (TRC) architecture that preserves recent interactions in full detail while compressing older interaction history into key-value representations. We evaluate our methods on three models with distinct architectures, assessing long-session stability, information retention, and computational efficiency. Across all three models, we show that LoRA adaptation mitigates performance degradation over ten-minute sessions, while adding TRC extends these gains beyond the training session length and reduces KV-cache growth by more than 90%.