Recurrent harness improves online learning for language model agents
Recurrent Harness for Agentic Continual Learning
Machine LearningArtificial IntelligenceComputation and Language
Summary
Language model agents often face changing information when interacting over time, making it hard to learn continuously without forgetting earlier knowledge or getting confused. The authors propose a new method called ReHarness that allows these agents to revise their past understanding by looking backward at their previous thoughts, improving how they adapt without needing extra storage or validation data. Their approach consistently performs better than existing methods across multiple tests and language models. This helps the agents keep better memory and flexibility while learning from ongoing streams of data.
What this means in practice
- •For chatbot developers: Improve chatbots to adapt continuously and revise previous responses without extra training data during long conversations.
- •For autonomous system engineers: Enhance autonomous agents to update their knowledge online and correct mistakes in changing environments more effectively.
Authors
Yuxin Xiao, Zhiyun Lu, Jianyu Wang, Yangtian Zhang, Yang Song, Qifan Wang, Bailin Wang
Abstract
Language model agents increasingly operate over long interaction streams, requiring online adaptation to distribution shifts through continual learning. However, existing agent harnesses typically either evolve their states only through forward information flow or rely on additional replay or validation data for harness optimization. To address this, we propose ReHarness, a recurrent harness framework for agentic continual learning that complements forward read and write with backward revision under a strictly online streaming protocol. We ask what information could have been captured differently in the harness state to better inform the current instance in the stream. Specifically, after each instance, we leverage the resulting trajectory as feedback to revise the reflections retrieved for that instance, thereby propagating feedback backward through the textual harness state. Across three continual learning benchmarks and three language model backbones, ReHarness consistently outperforms strong raw-context baselines, memory- and reflection-based methods, and prompt optimization approaches. Further analysis shows that ReHarness produces source-grounded and meaningful revisions, corrects overgeneralized or outdated experience under distribution shifts, and improves both stability and plasticity during continual learning.