Transformers revisit tokens with fresh feedback to improve language tasks
Write Back the $Δ$: Revisiting the Same Tokens with Fresh Representations
Machine Learning
Summary
Transformers, which process language step-by-step, usually don’t go back to improve earlier work once it’s done. The authors suggest a new way to let the model rethink its earlier guesses by feeding back only what new information it gained between steps, instead of repeating all the work or forcing fixed updates. They call this method ReFlux, which dynamically decides what to send back and can improve both understanding the same word again and processing sequences in order. Their experiments show this feedback helps language models better predict text and answer complex questions without extra heavy computation.
What this means in practice
- •For natural language processing engineers: Enhance language model accuracy on diverse datasets by enabling iterative refinement of token representations without increasing base computation.
- •For real-time speech recognition teams: Improve streaming token processing efficiency and accuracy by integrating dynamic feedback loops that revisit earlier representations without slowing down.
Authors
Wencheng Ye, Anning Hu, Xiangdong Zhang, Tianyi Wang, Yikang Li, Hengyu Jin, Bing Li, Junchi Yan
Abstract
Transformers process information strictly forward through depth, preventing deeper computation from revisiting and refining earlier representations. To augment the standard forward pass, existing approaches either re-execute depth, incurring additional computation, or modify the residual stream using predefined directions, limiting their instance-level adaptation. Recently, inference-time feedback offers a direct mechanism for recycling endogenously produced computation by writing deeper residual states back to earlier layers, yet what should be fed back remains unclear. We argue that the depth increment Delta, capturing newly accumulated computation between two layers, provides a more effective, composable, and scalable feedback signal than the full state. Building on this observation, we introduce ReFlux, a learnable feedback graph that dynamically selects and composes increment-carrying routes. ReFlux supports synchronous feedback to the same token and streaming feedback to subsequent tokens. Extensive experiments across various models, corpora, and benchmarks show that synchronous ReFlux consistently reduces perplexity across ten language-modeling corpora, and improves accuracy by 2.1-2.3 points, with gains reaching 4.7 points on multi-hop reasoning. Streaming ReFlux further retains most of these gains while preserving the base model's 1x theoretical backbone FLOPs. These results establish ReFlux as an efficient paradigm for unlocking the latent computational potential of LLMs, allowing them to revisit the same tokens with fresh representations. Code implementation can be found at https://github.com/gooogleshanghai/reflux.