EnvPilot improves software setup by learning from past solutions
EnvPilot: Systematic Design and Evaluation of an Experience-Augmented Agent for Software Environment Setup
Software Engineering
Summary
Setting up software environments can be complicated and usually needs expert help. The researchers created EnvPilot, a tool that learns from previous setup attempts to avoid repeating mistakes and improve over time. EnvPilot uses a memory of past experiences and picks the best strategies to offer precise guidance. They tested EnvPilot on many real software projects and found it works better than older methods. Their study shows that having a smart way to store and retrieve past knowledge really helps.
software environment setupexperience-augmented agenttrajectory-derived memorycontext-aware retrievalsoftware ecosystemsautomated setupbenchmarkpass@1 success rate
Authors
Hanwu Chen, Hanyu Lin, Zhanjiang Yang, Linhao Zhang, Aoyan Li, Jinxi Li, Meng Li, Yin Chen, Daoguang Zan
Abstract
Environment Setup is a critical yet complex task in software engineering that relies heavily on expert knowledge. Existing automated environment setup methods lack the ability to accumulate experience from past execution trajectories and to evolve over time. As a result, their performance is limited because they often perform redundant exploration, ignore useful past solutions, and fail to generalize across diverse software ecosystems. We present the systematic design and empirical validation of EnvPilot, an experience-augmented agent that operationalizes trajectory-derived experience reuse for software environment setup. EnvPilot maintains an expandable Trajectory-Derived Memory (TDM), initialized with 667 high-quality experiences. It systematically transforms implicit knowledge from historical execution trajectories into structured experience and retrieves the most relevant guidance during task execution through the Context-aware Retrieval mechanism. This enables EnvPilot to combine multiple validated setup strategies, providing more precise and detailed guidance than methods that rely solely on static project files or web retrieval. To evaluate EnvPilot, we construct AES-Bench, a multilingual benchmark of 112 real-world GitHub instances across 9 programming languages. Experiments show that EnvPilot achieves a new state-of-the-art (SOTA) with a 75.00% Pass@1 success rate while reducing reasoning costs. Our empirical study shows that both the structured experience representation and the Context-aware Retrieval mechanism are essential.