Self-Authored Verification Is Unreliable in Heuristic Self-Improving Agents
2026-07-27 • Computation and Language
Computation and LanguageMultiagent Systems
AI summaryⓘ
The authors study how agents that improve themselves by rewriting their own rules can mistakenly think they are getting better because they control their own tests. They show that this leads to a gap between what the agent believes about its performance and how it actually performs when deployed. To fix this, the authors propose SEAL, a method that uses an independent acceptance test the agent cannot influence, helping to catch real regressions. Their experiments show that SEAL reduces errors better than just relying on self-written tests, especially as agents get more capable.
self-improving agentspolicy rewritingself-verificationdeployment evaluationregressionheuristic learningtest harnessSEAL (Sealed Exogenous Acceptance Loop)agent capabilitytrial-and-error discovery
Authors
Diandian Guo, Cong Cao, Fangfang Yuan, Yingqi Wang, Yueshan Wang, Dakui Wang
Abstract
Self-improving agents accumulate capability by repeatedly rewriting procedural policies, controllers, or heuristic rules. They typically rely on self-authored tests or metrics to decide whether to accept subsequent edits. The agent controls both the optimized object and its verifier. As a result, self-assigned scores can remain near perfect while real deployment performance degrades or stays low. We study this problem through the verifier--deployment gap. This gap refers to the discrepancy between an agent's self-authored verification signal and a sealed deployment evaluation that the agent cannot observe or access. We ask how self-authored verification fails under iterative policy-and-test rewriting, how the failure changes with capability, and how little exogenous trust is sufficient to prevent real regressions from being deployed. To address this problem, we introduce a Sealed Exogenous Acceptance Loop (SEAL). SEAL retains self-authored tests but compares each candidate with the incumbent through a fixed harness-side audit. The agent cannot author or inspect the audit, receives only accept/reject, and the whole incumbent state is retained after a clear regression. Our experiments show that this problem often appears in heuristic learning settings. These settings require trial-and-error discovery of the target objective. We further find that failures of self-written verification are stratified by capability. Weaker agents tend to damage previously acquired strategies behind easy self-tests. Stronger agents are more stable, but they still mismeasure the deployment distribution. Standard self-written constraints do not reliably close this gap. In contrast, SEAL outperforms unprotected baselines across six models and three random seeds. Reliable self-improvement need not abandon self-verification, but it requires at least one deployment-acceptance signal outside the agent's control.