Routed continual learning improves retention and reduces costs for AI agents
Routed Continual Learning
Machine LearningMultiagent Systems
Summary
When AI agents face many different but related problems over time, they can learn better by remembering past experiences. The authors created a system called Routed Continual Learning that groups past experiences into clusters, each with its own way of storing information. Before solving a new problem, the system decides which cluster to use or whether to create a new one, making learning more efficient. Their tests showed that this method improves performance and significantly cuts costs compared to treating all experiences as one big pool.
What this means in practice
- •For machine learning engineers: Improve multi-task AI agents by dynamically selecting prior knowledge clusters and memory methods for each new task to boost performance and cut compute costs.
- •For customer support platform developers: Enhance virtual assistants that handle varied customer issues by routing past relevant interactions into efficient memory clusters to speed up problem solving.
Authors
Michael Philip, Mahesh Balakrishnan, David Geraghty, David Mao, Vidhya Venkat, Ilya Mironov
Abstract
Agents deployed at scale repeatedly encounter related problems, and it is natural to let them learn from each other's experience rather than starting over each time. Unfortunately, there is no single way to retain that experience that works well across settings, and no way to know in advance which future problems a given experience will help with. A global history preserves every detail but becomes expensive and mixes together experience that may not belong together; compressing that history into one fixed representation limits the cost but can throw away valuable details. We introduce Routed Continual Learning (Routed CL), an architecture that decides how to partition and retain experience. Past trajectories are grouped into clusters, each of which maintains state using one memory system from a fixed library. Before each run, an LLM-based router either reuses an existing cluster or creates a new one, basing its decision on the problem statement, memory system descriptions, and cluster summaries. We also introduce Context Memory Adapter, which mediates between the memory system and the agentic loop. We evaluate the architecture on a heterogeneous stream that interleaves 301 instances from the six CL-Bench tasks with 97 banking instances from $τ^3$-Bench. Across four language models and five matched rollouts, Routed Continual Learning raises normalized reward over a single global in-context learning (ICL) history by 15-30% of the maximum attainable normalized reward, achieves higher mean normalized reward than ICL-only clustering, and reduces estimated API cost by 65-72% relative to Global ICL. These results support approaching continual learning as a routing problem that chooses both a learning mechanism and a state cluster rather than committing to one representation globally.