Agents learn to improve reliability by remembering past mistakes
Closing the Consistency Gap: Self-Evolving Agents That Learn to Stay on Course
Artificial Intelligence
Summary
Large language model agents can often give the right answers but struggle to be consistently right every time they try the same task. The authors found that these agents only succeed on every one of five repeated attempts about half the time, even though they do well on average. To fix this, the authors created a system that helps agents spot where they go wrong and remember these trouble spots to avoid repeating the same mistakes. This approach improved the agents’ consistency in completing tasks successfully across repeated tries. The work aims to make AI agents more trustworthy by helping them stay on course.
large language model (LLM)agent trajectoryReAct agentconsistency gapepisodic memoryAppWorld benchmarkGuideline GeneratorConsistency Analyzertask generalization
Authors
Evelyn Duesterwald, Benjamin Elder, Lilian Ngweta, Shashanka Ubaru, Malgorzata Zimon
Abstract
Large language model (LLM)-powered agents can be accurate on average yet unreliable in production, a discrepancy that has been observed but remains largely unaddressed. When given the same task five times, a ReAct agent on the AppWorld benchmark using GPT-4.1 succeeds in all five runs only 53% of the time, even though its per-run pass rate averages 77%. We call this 24-point shortfall the consistency gap, and we argue that addressing it is a precondition for trustworthy AI agent deployment. We present a self-evolving agent framework that reduces this gap by identifying unstable, low-consistency steps in agent trajectories and converting them into episodic memory the agent can draw on in future runs. At its core is a Consistency Analyzer that pinpoints where and why a trajectory is likely to flip across executions, and a Guideline Generator that converts the diagnosis into targeted guidelines, committed to memory and injected into future agent executions on similar tasks. On AppWorld with ReAct/GPT-4.1, our framework raises the fraction of tasks that succeed in all five runs by +16 points on same-task evaluation and +13 points on similar-task generalization.