ExpVoyager improves agent skill building by navigating past experiences
ExpVoyager: Direct Experience Navigation for Dynamic Agent Skill Synthesis
Computation and Language
Summary
Teaching AI agents to learn from their past experiences helps them get better at new tasks. Traditional methods turn all past experiences into fixed skills too early, which can lose important useful information. The authors present ExpVoyager, a system where agents actively explore their previous experiences based on what they need now, allowing them to find and reuse helpful skills more effectively. This approach leads to better performance on tasks and adapts well as the agent gains more experiences over time.
What this means in practice
- •For ai system engineers: Build AI agents that access and reuse past experience dynamically to improve performance on evolving tasks.
- •For robotics developers: Enable robots to adapt skills on demand by navigating prior interaction histories rather than relying on fixed pre-learned routines.
Authors
Kwangwook Seo, Dongha Lee
Abstract
Learning from experience in LLM agents has become a key paradigm for developing self-evolving agents that continuously learn and expand their capabilities. Within this paradigm, synthesizing the agent skill has emerged as a promising solution for transforming accumulated experience into reusable procedural knowledge, serving as an important layer for the harness system that supplies agents at runtime. Despite its potential, existing approaches largely abstract past experience into fixed procedural knowledge before downstream demands are known, which risks discarding knowledge that later becomes critical while retaining instance-specific details irrelevant to future tasks. In this paper, we reframe agent skill synthesis as a dynamic navigation problem over past experience, where agents actively explore accumulated trajectories on demand for the current task with targeted and fine-grained access to experience knowledge. To this end, we propose ExpVoyager, a novel framework in which a skill curator navigates raw experience across different views and resolutions, continually identifying reusable procedural knowledge from what it observes while tracking remaining knowledge needs that guide where to navigate next. Extensive experiments demonstrate both the effectiveness and versatility of ExpVoyager, showing consistent improvements in downstream task performance, continual gains as the experience space scales, and practical compatibility with existing skills under efficient experience access.