Test-time learning improves reasoning by adjusting small bias parameters
Label-free steering: Compressing test-time reinforcement learning into bias-only subspaces
Machine LearningArtificial Intelligence
Summary
Some AI models can improve their reasoning while being used, without needing new labeled data. The authors studied whether this improvement could happen by changing only a tiny part of the model’s parameters, specifically bias settings, while keeping the main model fixed. They found that this small adjustment using label-free signals helped models perform better on math and reasoning tasks. Their method also worked when tested on new problems the model had never seen before. This means AI can adapt efficiently at test time by tweaking just a few internal settings.
What this means in practice
- •For ai development teams: Enable AI models to adapt and improve at test time by updating only small bias parameters without needing labeled data.
- •For software engineers in voice assistants: Improve audio reasoning capabilities dynamically during deployment by adjusting bias-only model components with label-free rewards.
Authors
Naveen Vakada, Mingyuan Li, Shaoxiong Ji
Abstract
Test-time reinforcement learning (TTRL) enables models to improve their reasoning without relying on labeled training data, but existing approaches typically optimize a large fraction of the model parameters. This raises a natural question: can effective test-time adaptation emerge when both the reward signal and the optimization space are severely restricted? We answer this question with label-free bias-only TTRL, which uses majority-vote pseudo-labels as rewards and optimizes only approximately 100K bias parameters while keeping the pretrained backbone frozen. On MATH-500, our approach reaches 76.67% accuracy, slightly exceeding our own labeled bias-steering reproduction while optimizing 76,000x fewer parameters than full-parameter TTRL. The same training procedure improves performance across vision-language and audio reasoning tasks, including MathVista, AI2D, LogicVista, and MMAU. We further show that the learned steering vectors transfer to 4,500 held-out MATH problems, indicating that the adaptation is not limited to the problems used during test-time optimization. Finally, we analyze why this highly restricted adaptation can work, showing that majority-vote reliability improves with rollout consensus and that bias subspaces with greater accessible gradient energy exhibit stronger downstream trainability. These results demonstrate that substantial test-time adaptation can emerge from optimizing a tiny bias-only subspace using entirely label-free rewards.