Agents struggle to learn and adapt reliably from real-world experience
ServeLearnBench: How Well Can Agents Self-Improve from Serving Experience?
Machine LearningArtificial Intelligence
Summary
AI agents that perform complex tasks often face environments where the rules or knowledge are hidden and keep changing. The researchers created a new benchmark, ServeLearnBench, that tests how well these agents can learn and improve by interacting with such evolving environments. They found that current methods still have big gaps: agents often fail to learn effectively from experience, can lose skills they've already mastered, and don’t explore enough to adapt well. This benchmark helps highlight these challenges and provides a way to track progress toward more reliable self-improving agents.
What this means in practice
- •For ai system developers: Evaluate and improve continual learning methods for agents deployed in dynamic real-world environments using ServeLearnBench.
- •For customer support engineers: Assess AI assistants’ ability to adapt from customer interaction data in domains like retail and banking through ServeLearnBench’s realistic tasks.
Authors
Haizhong Zheng, Yizhuo Di, Ranajoy Sadhukhan, Shuowei Jin, Beidi Chen
Abstract
Large language model agents are increasingly deployed to perform complex tasks in real-world environments. However, the knowledge required for correct behavior in these environments is often implicit, undisclosed, and subject to change over time. Recent continual-learning harnesses seek to address this challenge by enabling agents to improve from serving experience. Yet the effectiveness and limitations of these methods are not yet well characterized. Existing benchmarks provide only partial coverage: some explicitly provide the target knowledge, others assume a static environment, and those that support continual adaptation remain limited in scale and knowledge diversity. To enable systematic evaluation, we formalize an evolving-environment streaming dataset (EESD), in which agents must infer, apply, and revise latent environment knowledge from interaction and outcome feedback as hidden policies evolve, and introduce ServeLearnBench, spanning retail support, banking, and sales-pitch generation with 53 environment windows and 7,718 tasks. We evaluate five learning harnesses (RAG, Mem0, SkillOpt, Continual Harness, and Prime) across six models (GPT-5.6 Terra, Opus 5, Kimi K3, GLM-5.3, DeepSeek V4.1 Flash, and GLM-5.3 Flash), covering 28 model-harness pairs and 252 learning runs. Our evaluation reveals three main findings: a substantial gap remains between task capability and learning from experience; continual adaptation is costly and can degrade already-correct behavior; and insufficient exploration emerges as a key bottleneck to effective adaptation. Overall, ServeLearnBench provides a controlled testbed for diagnosing these limitations and tracking progress toward agents that continually and reliably improve through serving experience.