Static analysis helps AI improve code quality more deeply
From Cosmetic to Structural: A Paired Experiment on Metric-Guided Agentic Refactoring
Software Engineering
Summary
When AI tools fix code, they tend to make small, surface changes instead of the bigger improvements that reduce long-term problems. The authors tested whether showing the AI specific feedback about code problems helps it make better fixes. They found that giving the AI detailed code reports leads it to make deeper, more meaningful changes, but it can take longer and sometimes cause new issues. The AI also needs extra support when working with very poor quality code to avoid mistakes.
What this means in practice
- •For software maintenance teams: Use static-analysis feedback to guide AI tools for more effective code restructuring and technical debt reduction.
- •For competitive programming platforms: Integrate AI-driven code refactoring guided by quality scores to improve solution maintainability automatically.
Authors
Nadim Hagatulah, Markus Borg, Emma Söderberg
Abstract
The role of coding agents is expanding beyond writing new code toward maintaining existing systems as AI teammates. Yet recent studies show that agent refactorings favor superficial, consistency-oriented edits over structural transformations needed to resolve Technical Debt (TD). We investigate if explicit static-analysis feedback closes this gap. Through the CodeScene MCP server, we give an open-source agent (gptme) access to code smell reports and CodeHealth (CH) scores (1--10, higher is better), and compare guided against unguided refactorings in a paired experiment covering $24{,}620$ single-file competitive programming solutions in C++, Java, and Python. We find that guidance shifts the agent from performing cosmetic edits toward structural operations, raising the rate of effective refactorings by $21$--$26$ percentage points. The benefit depends on initial code quality, with language-dependent break-even points at CH $4.9$--$7.1$, below where the structural transformations pushed by guidance increasingly risk regression, indicating where the agent needs more support to refactor. Guided runs also shift the failure profile toward logical test failures, introduce new coupling smells, and consume $\sim$$3$--$5\times$ more tokens over $\sim$$2\times$ as many interaction turns. Our results indicate that AI teammates can benefit from similar static-analysis feedback that human developers already use, enabling structural transformations toward more effective TD remediation, at a higher token cost and more interaction turns.