Self evolving agents struggle to learn tasks without forgetting old ones
Are Self-Evolving Agents Continual Learners?
Artificial IntelligenceMachine Learning
Summary
Agents powered by large language models need to continually learn new tasks without forgetting what they already know, similar to how humans learn over time. The authors study agents that update themselves by changing their prompts and memories instead of retraining the entire model. They found that while this self-evolving approach helps in learning new things, it often causes the agents to forget earlier tasks or get confused when tasks are very different. This means current self-evolving agents are not yet good at lifelong learning without compromises.
What this means in practice
- •For software development teams: Improve AI assistants that adapt to changing project workflows by selectively updating prompts and memories without retraining models.
- •For customer support operations: Enhance chatbot systems to handle evolving customer queries by dynamically adjusting their knowledge without losing prior customer interaction understanding.
Authors
Xiaoyin Chen, Yizheng Huang, Yinglun Xu, Ziyun Zhao, Emmett McQuinn, Julie Liu, Hope Yao
Abstract
Large language model agents are often deployed in environments where new knowledge, tools, and workflows keep arriving. Such an agent is therefore expected to adapt continually, learning new tasks as they arrive without forgetting earlier ones. This resembles the continual learning problem, but at a pace that conventional gradient updates cannot sustain. Nonparametric self-evolving agents are a natural alternative: they adapt by rewriting their own harness, such as prompts, memory, and skills, from task feedback while the model weights stay frozen. We study whether such agents are effective continual learners across four drifting task streams that shift in topic, workflow, or domain (built from BIRD, BFCL, and ScienceWorld). Evaluating three representative methods spanning experience retrieval and self-modification (ReasoningBank, GEPA, and Meta-Harness), we find that although self-evolution improves aggregate performance on most streams, higher plasticity consistently comes at the cost of stability. Specifically, self-modifying methods apply global edits to a shared prompt or workspace, interfering with unseen tasks and overwriting earlier knowledge as the stream shifts. Conversely, similarity-based memory retrieval localizes updates across distant domains, but still retrieves foreign memories on unseen tasks and suffers from retrieval collision when similar queries require conflicting workflows. These findings show that current self-evolving agents remain incomplete continual learners.