EnvPilot improves software setup by reusing past experience
EnvPilot: Systematic Design and Evaluation of an Experience-Augmented Agent for Software Environment Setup
Software Engineering
Summary
Setting up software environments is often tricky and needs expert know-how. The authors created EnvPilot, a system that learns from past setup attempts to guide new ones more efficiently. It stores and retrieves useful setup experiences to avoid repeating mistakes and to work better across different programming languages. Tested on many real projects, EnvPilot was more successful and quicker than previous methods.
What this means in practice
- •For software development teams: Improve the efficiency and reliability of setting up diverse software environments by leveraging past setup successes and failures.
- •For devops engineers: Reduce time and errors in automated environment provisioning by using a system that adapts from prior setup experiences across languages.
Authors
Hanwu Chen, Hanyu Lin, Zhanjiang Yang, Linhao Zhang, Aoyan Li, Jinxi Li, Meng Li, Yin Chen, Daoguang Zan
Abstract
Environment Setup is a critical yet complex task in software engineering that relies heavily on expert knowledge. Existing automated environment setup methods lack the ability to accumulate experience from past execution trajectories and to evolve over time. As a result, their performance is limited because they often perform redundant exploration, ignore useful past solutions, and fail to generalize across diverse software ecosystems. We present the systematic design and empirical validation of EnvPilot, an experience-augmented agent that operationalizes trajectory-derived experience reuse for software environment setup. EnvPilot maintains an expandable Trajectory-Derived Memory (TDM), initialized with 667 high-quality experiences. It systematically transforms implicit knowledge from historical execution trajectories into structured experience and retrieves the most relevant guidance during task execution through the Context-aware Retrieval mechanism. This enables EnvPilot to combine multiple validated setup strategies, providing more precise and detailed guidance than methods that rely solely on static project files or web retrieval. To evaluate EnvPilot, we construct AES-Bench, a multilingual benchmark of 112 real-world GitHub instances across 9 programming languages. Experiments show that EnvPilot achieves a new state-of-the-art (SOTA) with a 75.00% Pass@1 success rate while reducing reasoning costs. Our empirical study shows that both the structured experience representation and the Context-aware Retrieval mechanism are essential.