GALA: Multimodal Graph Alignment for Bug Localization in Automated Program Repair
2026-04-09 • Software Engineering
Software Engineering
AI summaryⓘ
The authors propose GALA, a method to improve automated bug fixing when bugs are reported using both screenshots and code. Unlike previous methods that turn images into plain text and lose important layout information, GALA creates a graph to keep track of visual parts and their relationships. It then aligns this graph with the project’s code structure step-by-step, first finding relevant files and then pinpointing exact functions, before generating fixes. This approach helps connect what is seen in the UI directly to the code, leading to better bug localization and repair. Tests show GALA performs better than earlier methods on a multimodal benchmark.
Large Language ModelsAutomated Program RepairMultimodal LearningGraph AlignmentUI GraphCode LocalizationCall GraphPatch GenerationSemantic ConsistencyStructural Reasoning
Authors
Zhuoyao Liu, Zhengran Zeng, Shu-Dong Huang, Yang Liu, Shikun Zhang, Wei Ye
Abstract
Large Language Model (LLM)-based Automated Program Repair (APR) has shown strong potential on textual benchmarks, yet struggles in multimodal scenarios where bugs are reported with GUI screenshots. Existing methods typically convert images into plain text, which discards critical spatial relationships and causes a severe disconnect between visual observations and code components, leading localization to degrade into imprecise keyword matching. To bridge this gap, we propose GALA (Graph Alignment for Localization in APR), a framework that shifts multimodal APR from implicit semantic guessing to explicit structural reasoning. GALA operates in four stages: it first constructs an Image UI Graph to capture visual elements and their structural relationships; then performs file-level alignment by cross-referencing this UI graph with repository-level structures (e.g., file references) to locate candidate files; next conducts function-level alignment by reasoning over fine-grained code dependencies (e.g., call graphs) to precisely ground visual elements to corresponding code components; and finally performs patch generation within the grounded code context based on the aligned files and functions. By systematically enforcing both semantic and relational consistency across modalities, GALA establishes a highly accurate visual-to-code mapping. Evaluations on the SWE-bench Multimodal benchmark demonstrate that GALA achieves state-of-the-art performance, highlighting the effectiveness of hierarchical structural alignment.