Papers for

ai voice assistant developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Reframe reduces gpu load and speeds up live speech models

ReFrame: Silence-Aware Compute Reclamation for Full-Duplex Speech Models

Abstract: Full-duplex speech models execute dense model inference for every audio frame throughout a live interaction. This uniform framewise schedule creates sustained GPU load across dialogue states whose computation has strong, reusable temporal structure. ReFrame formulates full-duplex inference as a silence-aware computation problem. Mutual Silence Speculation (MSS) operates when both participants are silent: it evaluates multi-frame Temporal continuations, verifies text and hidden-state stability, and applies batched Mimi audio-token correction. Listening Frame Folding (LFF) operates while the user speaks and the agent listens: it processes already-arrived user frames through parallel Temporal execution and frame-batched audio-token generation while accepting only the verified prefix. Frame-Budget Reinvestment (FBR) assigns the released frame intervals to resumable external retrieval and language-model workers, then injects timely references through Moshi's conditioning interface. We instantiate ReFrame on five full-duplex models spanning Moshi-family, phase-structured, and flattened autoregressive generation. Across the complete 1,225-conversation FDBench, ReFrame reduces measured GPU execution time by 18.80-31.01% and retains 96.92-100% non-PAD text agreement on five different models. On HaluEval tests, FBR increases the number of correct generated spoken answers from 86 to 205 under the same auxiliary-service budget. ReFrame turns recurring dialogue structure into lower inference cost and higher end-to-end service capability.

Sun 1 NovMachine Learning
The gist
Full-duplex speech models process every audio frame continuously during a conversation, which uses a lot of computing power. The authors found that by detecting when both people are silent or when one speaks and the other listens, they can skip or batch some calculations without losing accuracy. Their system, ReFrame, smartly reallocates computing time saved this way to improve other tasks like retrieving information. This approach cuts GPU time by up to 31% while keeping almost all text outputs correct and also helps increase correct spoken answers.
Open → 2611.01835v1