Papers for

real-time speech recognition teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Reframe reduces gpu load and speeds up live speech models

ReFrame: Silence-Aware Compute Reclamation for Full-Duplex Speech Models

Abstract: Full-duplex speech models execute dense model inference for every audio frame throughout a live interaction. This uniform framewise schedule creates sustained GPU load across dialogue states whose computation has strong, reusable temporal structure. ReFrame formulates full-duplex inference as a silence-aware computation problem. Mutual Silence Speculation (MSS) operates when both participants are silent: it evaluates multi-frame Temporal continuations, verifies text and hidden-state stability, and applies batched Mimi audio-token correction. Listening Frame Folding (LFF) operates while the user speaks and the agent listens: it processes already-arrived user frames through parallel Temporal execution and frame-batched audio-token generation while accepting only the verified prefix. Frame-Budget Reinvestment (FBR) assigns the released frame intervals to resumable external retrieval and language-model workers, then injects timely references through Moshi's conditioning interface. We instantiate ReFrame on five full-duplex models spanning Moshi-family, phase-structured, and flattened autoregressive generation. Across the complete 1,225-conversation FDBench, ReFrame reduces measured GPU execution time by 18.80-31.01% and retains 96.92-100% non-PAD text agreement on five different models. On HaluEval tests, FBR increases the number of correct generated spoken answers from 86 to 205 under the same auxiliary-service budget. ReFrame turns recurring dialogue structure into lower inference cost and higher end-to-end service capability.

Sun 1 NovMachine Learning
The gist
Full-duplex speech models process every audio frame continuously during a conversation, which uses a lot of computing power. The authors found that by detecting when both people are silent or when one speaks and the other listens, they can skip or batch some calculations without losing accuracy. Their system, ReFrame, smartly reallocates computing time saved this way to improve other tasks like retrieving information. This approach cuts GPU time by up to 31% while keeping almost all text outputs correct and also helps increase correct spoken answers.
Open → 2611.01835v1

Transformers revisit tokens with fresh feedback to improve language tasks

Write Back the $Δ$: Revisiting the Same Tokens with Fresh Representations

Abstract: Transformers process information strictly forward through depth, preventing deeper computation from revisiting and refining earlier representations. To augment the standard forward pass, existing approaches either re-execute depth, incurring additional computation, or modify the residual stream using predefined directions, limiting their instance-level adaptation. Recently, inference-time feedback offers a direct mechanism for recycling endogenously produced computation by writing deeper residual states back to earlier layers, yet what should be fed back remains unclear. We argue that the depth increment Delta, capturing newly accumulated computation between two layers, provides a more effective, composable, and scalable feedback signal than the full state. Building on this observation, we introduce ReFlux, a learnable feedback graph that dynamically selects and composes increment-carrying routes. ReFlux supports synchronous feedback to the same token and streaming feedback to subsequent tokens. Extensive experiments across various models, corpora, and benchmarks show that synchronous ReFlux consistently reduces perplexity across ten language-modeling corpora, and improves accuracy by 2.1-2.3 points, with gains reaching 4.7 points on multi-hop reasoning. Streaming ReFlux further retains most of these gains while preserving the base model's 1x theoretical backbone FLOPs. These results establish ReFlux as an efficient paradigm for unlocking the latent computational potential of LLMs, allowing them to revisit the same tokens with fresh representations. Code implementation can be found at https://github.com/gooogleshanghai/reflux.

Sat 26 SeptMachine Learning
The gist
Transformers, which process language step-by-step, usually don’t go back to improve earlier work once it’s done. The authors suggest a new way to let the model rethink its earlier guesses by feeding back only what new information it gained between steps, instead of repeating all the work or forcing fixed updates. They call this method ReFlux, which dynamically decides what to send back and can improve both understanding the same word again and processing sequences in order. Their experiments show this feedback helps language models better predict text and answer complex questions without extra heavy computation.
Open → 2609.32457v1