Repository size influences references beyond code changes in pull requests
Reviewing Beyond the Change: Repository Size and Existing Code Across Agent and Human Pull Requests
Software Engineering
Summary
When people review code changes on platforms like GitHub, they sometimes mention parts of the project not directly changed in that update. The authors studied if this happens more in bigger projects and if it differs when the changes were made by automated tools versus humans. They found that in larger codebases, reviewers name existing parts outside the immediate change more often, and this pattern is similar whether a human or an automated system made the change. However, their tool likely missed some references, so the actual numbers might be higher.
What this means in practice
- •For software development teams: Improve code review tools to better highlight references to existing code outside changes in larger repositories, aiding reviewer understanding.
- •For software quality assurance teams: Design review processes that account for repository size effects when encouraging reviewers to reference existing code, improving review thoroughness.
Authors
Sabry E. Farrag
Abstract
Reviewers sometimes name existing repository items outside the change under review. We ask whether such references are more common in larger repositories, and whether the pattern differs between agent-authored and human-authored pull requests. An instrument resolves names in inline review comments against the base-commit repository; size-matched negative controls estimate chance matches. Among 92,042 comments on agent pull requests in 2,034 GitHub repositories, 6.5 per cent name an existing item outside the change. An internally pre-specified log-linear model finds an odds ratio of 1.35 per tenfold increase in code files (interval 1.15 to 1.59), and size-matched negative controls indicate that chance matching does not account for the rise. Exploratory analyses suggest that the association is concentrated from small to mid-sized repositories and is imprecise among large ones. Human pull requests in 1,845 of these repositories show a similar association. A pre-specified comparison finds a small agent excess just within the bound set as negligible, but the evidence does not establish a robust agent-specific difference. The instrument misses many references, so reported shares likely understate prevalence, and the validation cannot estimate how recall changes with repository size.