Writing Bug Reports for Software Repair Agents: What Information Matters Most?

2026-07-10Software Engineering

Software Engineering
AI summary

The authors studied how to write bug reports that help AI agents fix software problems better. They analyzed 441 real bug reports and identified different types of information in them, such as where the bug is located and suggested fixes. Using AI models, they found that information pointing directly to the code area or suggesting a fix makes the AI more successful at repairing the bug. Surprisingly, details like reproduction steps, which help human developers, are less helpful for AI agents. This means bug reports should focus more on guiding the AI where to look and what to try.

AI agentsbug reportlocalization cuessuggested fixesLLM backbonesbinomial regressionreproduction stepssoftware repairSWE-bench Verified
Authors
Vincenzo Luigi Bruno, Alessandro Giagnorio, Daniele Bifolco, Leon Wienges, Massimiliano Di Penta, Gabriele Bavota
Abstract
Software development is increasingly moving toward agentic-first workflows. This includes AI agents responsible for generating initial fixes for submitted issue reports. In this setting, issue reports are no longer merely documentation for human maintainers; they become the primary task specification for the agent. However, little is known about how such reports should be written to maximize the agent's chances of producing a correct fix. We study what makes a bug report agent-ready. Starting from the SWE-bench Verified benchmark (i.e., a collection of 500 real repository issues with human-written gold patches and test suites for evaluating generated fixes) we manually classify each issue by change type (e.g., bug fix vs refactoring) and annotate each sentence with its information type, such as observed behavior, expected behavior, reproduction steps, localization cues, and suggested fixes. We focus on the 441 issues representing bug reports, and we run on them mini-swe-agent using three LLM backbones (i.e., GPT-5-mini, MiniMax M2.5, and Gemini 3 Flash). We then fit a binomial regression model to estimate the incremental association between each information type and agent success, controlling for confounding factors. Our results suggest that agentic-first reports benefit most from information that narrows the agent's search and repair space. Localization cues, such as references to affected code areas, are positively associated with successful repairs, while suggested fixes, expressed either in code or natural language, show some of the strongest positive associations with pass probability. An ablation study removing selected information types confirms that agents benefit less from information traditionally useful to humans, such as reproduction steps, and more from sentences that expose a repair direction, either through bug localization or a suggested fix.