Comparing the Quality of Code Generated by Vibe Coding Tools
2026-08-17 • Software Engineering
Software Engineering
AI summaryⓘ
The authors studied how three popular AI tools generate code by making three web apps with each and checking the quality using SonarQube. They found that each tool had different strengths and weaknesses in the code, such as the number and seriousness of problems and how complex the code was. One tool, Lovable, had fewer serious issues but more minor code problems, while the other two tools made code with more serious issues. This means that picking an AI coding tool involves trade-offs in code quality beyond just how fast it works.
AI code generationstatic analysisSonarQubecode qualitycode smellscyclomatic complexitycognitive complexitycode duplicationseverity distributionremediation effort
Authors
Gustavo da Mota, Kiev Gama
Abstract
The use of AI agents for automatic code generation has become increasingly common in software development. However, concerns remain about the quality of the generated code, including aspects of maintainability, readability, and long-term evolution. This study compares the structural quality of code produced by three widely adopted vibe coding tools --- Lovable, v0, and Replit --- starting from a single generation prompt. We generate three independent projects per tool, totalling nine web applications, and submit them to static analysis with SonarQube. We collect metrics such as the number of issues, severity distribution, estimated remediation effort, cyclomatic and cognitive complexity, and code duplication. Preliminary results show that the tools exhibit distinct qualitative profiles: Lovable concentrates issues of lower severity but presents a substantially higher density of code smells per KLOC, while v0 and Replit produce more code with more aggressive severity profiles. These findings suggest that choosing between vibe coding tools involves structural trade-offs that go beyond perceived productivity.