Selective verification boosts coding agents accuracy with less cost
SWE-VERIFIER: Selective Verification for Cost-Efficient Coding Agents
Machine LearningArtificial Intelligence
Summary
Coding agents that write or generate code often spend a lot of computing power trying out different actions, which can be expensive and slow. The authors created SWE-Verifier, a small model that decides when the agent actually needs to check more options, saving effort by not verifying every single step. By using this selective approach, coding agents make better decisions more efficiently, improving accuracy on tested benchmarks while only slightly increasing costs. SWE-Verifier works well even on tasks it wasn't specifically trained for, showing it can help different coding problems.
What this means in practice
- •For software development teams: Reduce cloud API costs by selectively verifying coding suggestions only when needed, improving automated code generation accuracy efficiently.
- •For ai service providers: Integrate lightweight verification modules like SWE-Verifier to lower compute expenses while maintaining or improving response quality in coding assistants.$Commercial implications: Enables development of cost-effective coding assistant services that balance performance and operational costs at scale.
Authors
Kaijian Zou, Vaibhav Chennareddy, Yunxiang Zhang, Naihao Deng, Longju Bai, Jiaxin Pei, Rada Mihalcea, Lu Wang
Abstract
Coding agents spend substantial computation generating actions and processing long interaction histories. Poor action choices can steer agents into low-value states, wasting computation and increasing API costs. Sampling and verifying alternative actions can improve the agent's decisions, but applying this procedure at every step adds substantial overhead. We introduce SWE-Verifier, a lightweight generative model that selectively triggers candidate sampling and verification. It scores the coding agent's proposed action to determine when additional candidate sampling is needed, then ranks the resulting actions before execution. We train SWE-Verifier from Qwen3.5-4B using preference data that groups contrasting actions with their resulting environment observations and rubric-guided teacher scores. Using the mini-swe-agent harness, SWE-Verifier improves resolution from 75.47% to 77.20% on MiniMax-M2.7 at 1.09 times cost and generalizes to DeepSeek-V4-Flash without training on its actions, improving resolution from 72.60% to 74.27% at 1.13 times cost. Across both models, SWE-Verifier verifies fewer than 4% of steps and achieves comparable or higher resolution than the trajectory and all-step baselines at substantially lower cost. These results demonstrate that selective candidate sampling and lightweight verification provide complementary gains in coding-agent performance and inference efficiency. Our model, code, and data are available at https://github.com/zkjzou/SWE-Verifier.