Evaluating the Impact of Explainable AI on Trust in AI-Assisted Code Review
2026-07-27 • Software Engineering
Software EngineeringArtificial IntelligenceHuman-Computer Interaction
AI summaryⓘ
The authors studied how explanations from AI tools affect developers' trust in AI-generated code reviews. They tested three setups: full explanations, just feedback without explanation, and no explanation. They found that full explanations made developers trust the AI more, but they didn't always agree with its suggestions. When explanations were moderate, agreement was highest, possibly because full explanations made developers think more critically. No explanations led to the lowest trust and agreement, showing that some explanation helps developers make better use of AI in code review.
Large Language ModelsExplainable AICode ReviewTrust in AIAI-assisted DevelopmentHuman-AI InteractionUser StudySoftware EngineeringAI Recommendations
Authors
Zhenhan Gao, Marvin Muñoz Barón, Umm-e Habiba, Daniel Graziotin, Stefan Wagner
Abstract
Background: Large language models (LLMs) are increasingly used to automate code review, but the reasoning behind their decisions remains hard to understand. Developers struggle to assess the validity of LLM-generated reviews, making it difficult to gauge how much trust to place in them. The role of Explainable AI (XAI) in code review and its impact on trust remain underexplored. Objective: We study the influence of XAI on developer trust in AI-assisted code reviews. Method: We conducted a within-subjects user study with 34 participants, comparing three LLM-based code review systems with varying levels of XAI support: Condition A (detailed explanation and review feedback), Condition B (review feedback only), and Condition C (no explanations). Participants reviewed real-world code change requests alongside the AI-generated reviews. We measured trust perceptions, agreement with the AI recommendation, the reasoning given for each decision, and the time taken. Results: The level of explanation significantly influences both trust and agreement with AI recommendations, but in different ways. Full explanations (A) yield the highest perceived trust (M = 3.99/5) but not the highest agreement, whereas moderate explanations (B) achieve the highest agreement (89.22%). This could suggest that more explanation prompts developers to question AI recommendations more frequently. No explanations (C) results in the lowest trust and agreement. Explanation level did not significantly affect review time. The most commonly cited reasons for decisions were code readability and correctness. Conclusion: Incorporating XAI into code review significantly changes trust perceptions and agreement with AI recommendations. These results inform the design and evaluation of trustworthy AI-based code review systems, as well as studies on the human factors of AI-assisted software development.