Models learn to decide when and how to improve at test time
Agentic-TTT: Training test-time policy for test-time training
Machine LearningArtificial IntelligenceComputation and Language
Summary
Sometimes, language models can get better by learning while they are used, called test-time training (TTT). But not every way of learning on the spot works well all the time, and picking the wrong method can waste time or harm performance. This paper presents Agentic-TTT, which lets a model figure out when to learn, what learning method to use, and whether to reuse previous skills, making it smarter about improving itself. The authors show this method nearly doubles the performance gains and can work on new tasks it didn't see before.
What this means in practice
- •For machine learning engineers: Improve deployed language models by automatically selecting when and how to update model parameters based on incoming data.
- •For ai system operators: Reduce runtime compute by dynamically deciding if further model adaptation is worthwhile during live use.
Authors
Jiahao Lu, Mohan Kankanhalli
Abstract
Test-time training (TTT) adapts an LLM's parameters using signals derived from test inputs, and can make striking improvements in pre-specified settings such as IMO competitions or designated open problems. By turning deployment experience into parameter updates, TTT provides a direct mechanism for model-level self-improvement. Yet TTT is not universally beneficial: each TTT algorithm works in different settings, and applying an ill-suited method could waste test-time compute or even damage model performance. Therefore, such parameter-level self-improvement requires agency: the model must decide when TTT is warranted, which algorithm to invoke, and whether an existing skill can be reused. To fill this gap, we introduce Agentic-TTT, which learns a test-time policy to govern those decisions. Agentic-TTT turns TTT procedures into callable tools, treats accumulated skills as an evolving deployment environment, and trains its policy using the observed utility gains from its decisions. On our benchmark, Agentic-TTT nearly doubles the utility over the backbone model, learns to trade off utility against compute, and generalizes to domains unseen during training. Together, these results point toward autonomous self-improvement: models that can decide how to learn from their own deployment experience.