Full-duplex speech models cut memory use with text compression
Acoustic-to-Text KV Compression for Full-Duplex Speech Models
SoundArtificial Intelligence
Summary
Speech models that listen and talk at the same time build up lots of data in memory, making long conversations hard to handle. The authors propose converting spoken audio into compact text during pauses, saving memory by removing old audio data but keeping recent speech and transcripts. They train this system to maintain the model’s original behavior using special learning techniques. This method cuts memory usage by over 60% in tests and improves tasks like transcription and answering questions from speech. It also keeps smooth conversation flow, handling pauses and interruptions well.
What this means in practice
- •For speech recognition developers: Reduce memory needs for live speech-to-text models by compressing audio input into text efficiently during conversation gaps.
- •For voice assistant engineers: Improve long interaction handling in assistants by lowering memory use while keeping smooth turn-taking and quick responses.
Authors
Yejin Lee, Seungbeom Kim, Yongha Lee, Kyuhong Shim
Abstract
Full-duplex speech language models continuously accumulate acoustic key-value (KV) states, making long-running interactions memory-intensive. During listening, the model can finish processing an audio unit before the next arrives; we term the remaining interval listening-time slack. We propose acoustic-to-text KV compression, which introduces a transcription side channel to convert incoming speech into compact textual memory within this interval. When the cache exceeds a target budget during inference, older acoustic states are evicted while transcripts and recent acoustic context remain. We train the side channel with LoRA using cross-entropy on transcription segments. To preserve listening and speaking behavior, we apply knowledge distillation to the original model's token-level output distributions at native prediction positions. On ten-minute LongSpeech sessions, our MiniCPM-o 4.5 implementation reduces peak streaming KV-cache size by 64.6% compared with the same model without eviction. The proposed method also improves transcription, temporal question answering, and summarization over the baseline. Full-Duplex-Bench evaluations further show comparable pause-handling, turn-taking, and interruption performance.