Arctic-SWE improves long-horizon AI training with model and system design

Arctic-SWE: Revisiting the Long Horizon Agent Training by Modeling-System Co-Design

Machine LearningArtificial IntelligenceComputation and Language

Summary

Training AI agents to perform tasks over long periods is hard to do well and hard to measure. The authors created Arctic-SWE, a system that combines known techniques in new ways to form a strong starting point for AI training, making it easier to test and improve new methods. They tested it by training large language models on coding tasks and showed it matches or beats other systems using less training effort. They also improved the training data and tests to better measure real progress. Their work helps others build better AI systems by providing a solid, reproducible baseline.

What this means in practice

  • For machine learning engineers: Train large language models more efficiently for complex coding tasks using a reproducible baseline that integrates established training techniques.
  • For ai platform developers: Build RL training platforms that combine synchronized policies and consistent state updating to improve training stability and efficiency.

Authors

Boyi Liu, Yite Wang, Xinyu Lian, Fan Shu, Sida Li, Feng Yan, Mingjia Zhang, Yuxiong He, Zhewei Yao

Abstract

Long-horizon agent training, including software engineering, is advancing rapidly, but the recipe and infrastructure needed for a strong reinforcement-learning baseline remain difficult to reproduce. This makes gains from new algorithms or data hard to distinguish from improvements to an under-tuned baseline. We present Arctic-SWE, a fully asynchronous RL framework that establishes such a baseline through model-system co-design, using overfitting on a small task pool as the testbed. This testbed exposes failures that early reward gains conceal and guides the integration of established components: gated reward, importance-weight control, aligned training and inference engines, and consistent state across policy updates. No single component is new, yet together they form the missing foundation. On raw R2E-Gym data, without new algorithms or data work, Arctic-SWE trains Qwen3.5-4B to 62.0% pass@1 on SWE-bench Verified. This is comparable to concurrent FrogNano (61.5%), which relies on online task synthesis, after about one-sixth as many training rollouts. Building on this foundation, we further filter and refine R2E-Gym, SWE-rebench V2, and Scale-SWE. With this curated subset, the 4B model improves from 47.4%/25.0%/37.9% to 65.0%/38.3%/59.7% on SWE-bench Verified/Pro-v1/Pro-v2, and the recipe also improves Qwen3.5-9B from 56.2%/31.9%/48.8% to 65.7%/39.6%/70.2% on Verified/Pro-v1/Pro-v2. Controlled comparisons show that both curation and test strengthening improve held-out performance, although stronger tests reduce apparent training-task mastery. Arctic-SWE provides a documented starting point for studying better algorithms, data and environments for asynchronous agent RL.