AI-assisted software does not make code more similar or lower quality

Does AI-Assisted Development Homogenize Software? An Empirical Study of Code Similarity, Quality, and Evolution

Software EngineeringArtificial Intelligence

Summary

People worry that using AI tools to write software might make all code look the same or be worse. The authors studied many long-lasting software projects in Java and Python over time, including before and after AI tools started helping. They found that for Java, code actually became less similar, and Python stayed about the same. The quality of software stayed steady or even improved slightly during the AI era. This means AI tools don’t necessarily cause software to be more alike or worse.

What this means in practice

Authors

Luciano Marchezan, Romain Lefeuvre, Kevin Delcourt, Benoit Combemale, Houari Sahraoui

Abstract

AI-assisted software development is evolving from interactive LLM assistance toward agentic systems that autonomously perform software engineering tasks, yet the effects of this shift remain insufficiently understood. We present an empirical study of 50 long-lived Java and Python open-source projects across three eras: pre-LLM, potentially LLM-assisted, and detected agentic development. Using snapshots from Software Heritage, we examine code similarity, software quality, and software evolution. Similarity is measured semantically through embedding-based cosine similarity and syntactically through CodeBLEU*. Quality is assessed with SonarQube metrics covering maintainability, complexity, reliability, and security. Evolution is characterized by annualized growth rates of LOC, functions, classes, files, and statements, and by commit activity in the three months preceding each snapshot. Results show no consistent evidence of increased similarity as semantic and syntactic similarity decreased significantly in Java for both the LLM and agentic eras relative to the pre-LLM era, while Python showed no significant differences. Software quality remained largely stable, although the agentic era showed significantly lower code-smell and SQALE index densities. Structural growth was generally lower in the agentic era, particularly for files, whereas development activity increased moderately in Java, especially in commits and added lines. These findings challenge the assumption that LLM- and agent-assisted development necessarily homogenizes code, degrades quality, or accelerates growth.