Systematic code reasoning boosts agent success in fixing bugs
Enforcing Code Reasoning in Agentic Scaffold for More Effective Issue Resolution
Software Engineering
Summary
Fixing software bugs often requires careful thinking about how the code works, but many automatic tools do this in a shallow or incomplete way. The authors studied how existing programming agents process bug fixes and found they frequently miss deeper reasoning steps. To improve this, they designed DeskCheck, a tool that guides agents to thoroughly analyze code, test fixes, and understand corner cases. DeskCheck led to more accurate bug fixes across multiple programming languages and large test sets, showing that structured reasoning helps programming agents work better.
What this means in practice
- •For software development teams: Improve automated bug fixing tools by integrating systematic code reasoning to enhance accuracy and reduce missed issues in software projects.
- •For quality assurance teams: Use better patch auditing and test generation guided by code reasoning to raise software reliability and catch corner cases before release.
Authors
Changshu Liu, Ji young Kim, Alireza Ghazanfari, Saanvi Nallu, Reyhaneh Jabbarvand
Abstract
Reasoning is central to ReAct-based programming agents, yet whether and how they reason about code remains unclear. We study 7,955 issue-resolution trajectories from six single- and multi-agent scaffolds across four programming languages on SWE-bench-Verified, SWE-bench-Pro, and SWE-bench-Lite. We identify 12 forms of code reasoning in 66.95% of trajectories. Code reasoning is associated with higher resolution rates, and its intensity increases with task complexity. However, it remains largely unguided: agents may focus on surface-level symptoms during localization, fail to refine near-correct patches, overlook issue requirements, and generate tests with limited behavioral diversity. To address these limitations, we propose DeskCheck, an agentic scaffold that systematically guides code reasoning throughout repository-level issue resolution. DeskCheck simulates execution paths to localize root causes, simulates candidate patches to refine near-misses, audits patches against issue requirements, and reasons about corner-case inputs and expected outputs before generating tests. On 731 SWE-bench-Pro instances, DeskCheck improves resolution rates over mini-SWE-Agent and Claude Code by 6.29% to 10.53%, at an additional cost of $0.02-$0.28 per instance (average: $0.14). Per-phase analyses show higher method-level localization recall, more successful repairs of imprecisely localized issues, and better detection of missed corner cases in near-correct patches. Exact McNemar tests show statistically significant improvements across all subject LLMs (p < 10^-10), while repeated experiments support robustness to agent nondeterminism. These findings motivate systematic code reasoning as a means of improving programming agents.