Software engineering agents learn better from milestone feedback signals
SWE-MILE: Asynchronous Potential-Induced Milestone Credit Assignment for Long-Horizon Software Engineering Agents
Artificial IntelligenceSoftware Engineering
Summary
Training software agents to improve long tasks is hard because they usually only get feedback at the very end. The authors propose SWE-MILE, a way for agents to get useful hints during the task by measuring progress on key files and tests. This approach lets the agent know which steps help or hurt overall progress without needing extra human evaluation. They also introduce a system to check changes quickly in parallel, making the feedback timely. Their tests show this leads to better agent performance on challenging software tasks.
What this means in practice
- •For software development teams: Improve automated code generation agents by providing real-time feedback on key development milestones during long and complex coding tasks.
- •For devops engineers: Accelerate validation of code changes by running parallel sandboxed verification checks that reduce latency in software deployment pipelines.
Authors
Chaoqun Cui, Hao Zhou, Meiqi Chen, Fandong Meng, Wenji Mao
Abstract
Long-horizon software engineering (SWE) agents trained with reinforcement learning with verifiable rewards (RLVR) typically receive only terminal outcome supervision, making it difficult to distinguish productive actions from redundant exploration or functional regressions. We propose SWE-MILE, an asynchronous potential-induced milestone credit assignment framework that derives fine-grained process supervision from workflow runtime, without auxiliary reward models or external evaluators. SWE-MILE quantifies task-relevant file exposure and test-state alignment as navigation and verification potentials, respectively. Differences in these potentials attribute milestone progress and regressions to individual actions, while discounted backward credit propagates supervision to preceding steps. To efficiently acquire intermediate verification states, SWE-MILE further introduces asynchronous shadow probing, which replays repository-changing actions in an isolated sandbox and runs verification in parallel with the agent's primary interaction, largely hiding verification latency. The resulting process credit augments terminal outcome advantages and provides informative learning signals. Experiments on two representative long-horizon SWE tasks demonstrate substantial improvements in agent performance, highlighting workflow runtime signals as a practical source of process supervision for long-horizon SWE agents.