Papers for

software testing specialists

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Fault localization helps but does not guarantee better AI bug fixes

From Finding to Fixing: How Useful is Fault Localization for Agentic Program Repair?

Abstract: Fault localization (FL) aims to identify code locations responsible for software failures and is commonly evaluated by how accurately it ranks known fault locations. Large language model (LLM) agents are changing program repair and are increasingly used to search and reason over repositories. Although recent LLM-based FL methods have improved localization accuracy, their impact on agentic repair remains unclear. To address this gap, we conduct a systematic empirical study of FL in agentic repair. We evaluate six recent FL methods on SWE-bench Verified and Defects4J with three LLMs under a common repair framework and analyze 16,248 repair trajectories. We also examine gold locations and bug-cause descriptions as forms of FL guidance across different repair contexts. The studied FL methods do not consistently improve repair success, and higher localization accuracy does not consistently lead to larger repair gains. Yet FL guides agents' early search and helps them reach relevant code sooner. Accurate locations can reduce repair token use by up to 39.2\%, but current FL methods often consume more tokens than they save during repair. Experiments with gold guidance suggest that FL has greater potential when bug inputs contain fewer location clues. Providing bug causes can further improve repair success beyond locations alone. We also find that ground truth from a single reference patch does not fully capture the locations used by successful repairs. These findings help clarify the value of FL in agentic repair and inform future work on its design and evaluation.

Mon 19 OctSoftware Engineering
The gist
Finding the exact spots in code that cause bugs is important for fixing software. This study looks at how useful spotting these fault locations is when AI methods try to fix code by themselves. The researchers found that knowing where the bugs are can help AI fix code more efficiently, but just being better at locating faults doesn't always mean more bugs get fixed. Also, giving AI explanations about the cause of the bug helps even more than just showing the spot. This means fault localization is helpful but has limits in improving AI-driven software repair.
Open → 2610.25826v1