How language models refine answers through repeated updates
Prediction Dynamics in Depth-Recurrent Language Models
Computation and LanguageMachine Learning
Summary
Language models can improve their predictions by repeatedly updating internal information as they work through a problem. The authors studied why intermediate guesses can seem right even while their confidence levels keep changing. They broke down these changes into parts related to common shifts, directions relative to the top choice, and how competitors’ scores change compared to the best answer. Their findings help explain how models maintain stable answers before finishing all updates by looking at the geometry of score changes.
What this means in practice
- •For language model developers: Improve prediction efficiency by identifying when intermediate model outputs already align with final answers.
- •For natural language processing engineers: Design better iterative refinement strategies for language models by understanding geometric score change patterns.
Authors
Xinyue Luo, Fei Yu
Abstract
Depth-recurrent language models refine predictions through repeated latent updates. Why can intermediate answers agree with the endpoint while their scores continue to change? We derive a sharp margin characterization that decomposes the conservatism of a magnitude bound into common translation, direction relative to the winner, and the pairing of each competitor's update with its score gap. Across Huginn-3.5B and Ouro-1.4B, accounting for update direction and competitor pairing reduces the mean earliest qualifying depth by a further 22.5-34.4% of the total depth beyond translation removal under full answer-text scoring. This retrospective comparison uses completed trajectories. Substantial contributions also occur under label scoring. For shared predictive distributions, we separate common and contrast motion orthogonally and express the common component through candidate-set mass and within-set concentration. Common and contrast energies can attenuate at different rates, allowing a growing preference-change share to coexist with shrinking absolute updates. These findings explain finite-depth answer preservation through the geometry and composition of observed score changes.