Product evaluation dominates over process audits in self improving AI studies
Auditing the Product, Not the Recursion: The Evaluation Evidence Behind Recursive Self-Improvement Claims
Machine LearningArtificial IntelligenceComputation and LanguageSoftware Engineering
Summary
Many studies claim artificial intelligence systems can improve themselves repeatedly, but often test only the final outcomes, not the process of improving. The authors looked at over 100 studies and found that most checked the system’s final performance but rarely repeated or thoroughly tested the entire improvement cycle. They also found that comparing earlier and later versions of improvement methods is uncommon. This suggests that while AI systems seem better, evidence for how reliably they improve themselves or how the improvement process works is still limited.
What this means in practice
- •For ai development teams: Use claim-indexed reporting standards to rigorously link AI self-improvement claims to appropriate evaluation methods, enhancing reliability and reproducibility.
- •For software testing engineers: Design testing protocols that audit repeated runs of AI improvement loops to better assess stability and process improvements over time.
A survey. It maps existing work.
Authors
Zihan Chen, Di Zhu, Lei Nico Zheng
Abstract
Recursive self-improvement (RSI) promises AI systems, such as self-evolving LLM agents, that become better at producing further improvements. Evaluating this promise requires separating product claims about the final system from process claims about the improvement process itself. Existing surveys specify the evidence these claims require, but how often studies report that evidence remains unmeasured. We examine this question through full-text coding of 114 eligible studies in a reproducible arXiv search frame frozen in October 2026, including 90 that describe their own systems as RSI. The codebook links claims to evaluation designs. Independent human coding, third-reader adjudication, and a separate blind coding of improver comparisons assess coding reliability. Among the 90 RSI studies, 90% report product evaluation beyond development tasks under a broad criterion that includes partial or unstated separation, while 60% report evaluation on tasks unused for selection. Only 22% audit two or more reruns of the full loop. In a human-coded random sample, the RSI studies show the same contrast: 45% evaluate on tasks unused for selection and 14% audit two or more reruns. Of the 26 studies claiming a better improvement procedure, 23 report a related comparison, but only 4 compare earlier and later procedures from identical starting artifacts at a matched budget. Across these results and a comparison set of established works, evidence for repeatability and improved procedures remains less developed than product evaluation. Three propositions explain how selection effects, shared evaluator errors, and changes in system state limit what product evaluations can establish. Together, the findings support a claim-indexed reporting standard that connects each claim to the evaluation design needed to test it.