Transformers revisit tokens with fresh feedback to improve language tasks

Write Back the $Δ$: Revisiting the Same Tokens with Fresh Representations

Machine Learning

Summary

Transformers, which process language step-by-step, usually don’t go back to improve earlier work once it’s done. The authors suggest a new way to let the model rethink its earlier guesses by feeding back only what new information it gained between steps, instead of repeating all the work or forcing fixed updates. They call this method ReFlux, which dynamically decides what to send back and can improve both understanding the same word again and processing sequences in order. Their experiments show this feedback helps language models better predict text and answer complex questions without extra heavy computation.

What this means in practice

Authors

Wencheng Ye, Anning Hu, Xiangdong Zhang, Tianyi Wang, Yikang Li, Hengyu Jin, Bing Li, Junchi Yan

Abstract

Transformers process information strictly forward through depth, preventing deeper computation from revisiting and refining earlier representations. To augment the standard forward pass, existing approaches either re-execute depth, incurring additional computation, or modify the residual stream using predefined directions, limiting their instance-level adaptation. Recently, inference-time feedback offers a direct mechanism for recycling endogenously produced computation by writing deeper residual states back to earlier layers, yet what should be fed back remains unclear. We argue that the depth increment Delta, capturing newly accumulated computation between two layers, provides a more effective, composable, and scalable feedback signal than the full state. Building on this observation, we introduce ReFlux, a learnable feedback graph that dynamically selects and composes increment-carrying routes. ReFlux supports synchronous feedback to the same token and streaming feedback to subsequent tokens. Extensive experiments across various models, corpora, and benchmarks show that synchronous ReFlux consistently reduces perplexity across ten language-modeling corpora, and improves accuracy by 2.1-2.3 points, with gains reaching 4.7 points on multi-hop reasoning. Streaming ReFlux further retains most of these gains while preserving the base model's 1x theoretical backbone FLOPs. These results establish ReFlux as an efficient paradigm for unlocking the latent computational potential of LLMs, allowing them to revisit the same tokens with fresh representations. Code implementation can be found at https://github.com/gooogleshanghai/reflux.