Scaling coding agents with source code based rl tasks improves performance
CodeMidas: Scaling Agentic Coding RL Environments from Code Itself
Artificial Intelligence
Summary
Training AI agents to write and fix code well needs lots of different programming tasks with ways to check their work. The authors created CodeMidas, a method that automatically builds such tasks directly from existing open-source code without relying on extra project info. This gives thousands of coding tasks covering many languages and areas. Training on these tasks helps AI coding agents do better on lots of programming challenges. The AI also learns smarter ways to explore code and check its own solutions.
What this means in practice
- •For software engineering teams: Build more effective automated coding assistants that improve issue repair and code generation by training on diverse tasks derived from existing code.
- •For automation engineers: Use scalable RL environments derived from real code to develop agents that can autonomously explore and fix software projects.
Authors
Bowen Ye, Lei Li, Shicheng Li, Zihao Yue, Linghao Zhang, Hanglong Lv, Yuanxin Liu, Wenhan Ma, Hao Tian, Rang Li, Jinhao Dong, Yikai Zhao, Xiangwei Deng, Hailin Zhang, Liang Zhao, Qi Liu, Lingpeng Kong, Tong Yang, Fuli Luo
Abstract
Training capable coding agents via reinforcement learning (RL) requires diverse tasks with reliable verifiers. Open-source codebases offer a rich source of such tasks, while existing methods typically rely on development artifacts such as issues and commits, limiting the range of tasks that can be extracted. To better scale RL environments, we present CodeMidas, an agentic pipeline that turns implemented functionality in existing codebases into executable RL environments using source code as its only task-specific input. CodeMidas allocates agentic compute to every stage of environment construction: agents explore implemented functionality to formulate behavioral specifications, construct tests grounded in execution of the original code, and validate and filter candidate tasks through execution checks and repeated solution rollouts. The resulting dataset has 5,545 training tasks from 3,185 open-source codebases spanning 23 programming languages and 15 technical domains. Training MiMo-V2.5 on these tasks with GRPO improves performance on all five diverse benchmarks, covering issue repair (DeepSWE + 11.7%), whole-program construction (ProgramBench +17%), and terminal work (Terminal-Bench v2.1 +8.5%). Ablations show that increasing the number of high-quality training tasks improves performance. Trajectory analysis shows the RL-trained agent demonstrates better behaviors like increasing codebase exploration and more diverse self-verification. These results establish source code as a scalable foundation for constructing RL environments that improve coding agents across diverse software tasks.