Self improving agents lose safety safeguards when rules overlap

Masked Erosion: How Acceptance Gates Lose Redundant Safeguards in Self-Improving Agents

Artificial Intelligence

Summary

When large language model agents improve themselves, they test changes using a gate that looks only at the overall success of tasks, not at hidden safety safeguards behind the scenes. The authors found that if several safeguards cover the same risk, deleting one can seem harmless because the others still work, which hides the problem. Over many tests, they showed how this ‘masking’ causes agents to lose important overlapping safety checks, making them vulnerable later if a weaker model is used. The paper also shows that traditional audits don’t catch this loss, but freezing safety details or testing one safeguard at a time can help.

What this means in practice

  • For ai safety engineers: Improve testing procedures to detect and maintain overlapping safeguards in self-improving AI agents to prevent hidden safety loss.
  • For machine learning platform operators: Design update audits that check single and combined safeguard deletions to ensure robust protection under evolving model changes.

Authors

Hau-Sing So, Daixian Liu, Weicai Long

Abstract

Self-improving LLM agents edit their own harness, the prompts and rules around a fixed model, and keep an edit only if an acceptance gate scores it higher on a batch of tasks. Such a gate sees what the harness does under the current model, not what it would still prevent if one safeguard failed. When safeguards overlap, deleting one is masked: the others still block the harm, so the gate decides on task success alone and redundancy erodes unseen. We measure harm for every subset of four written safeguards in two agent environments and run 411 improvement loops under ten gate rules, mostly as controlled deletion experiments, plus three runs of the GEPA optimizer. A deletion behind another working safeguard adds almost no observable harm. A paired significance test blocks deletions that pass by chance but not deletions that genuinely raise task success, and a larger evaluation batch lets it delete cheaper safeguards: at the same cost, 20-task batches kept every harness protected while 40-task batches lost all six. Large harm audits prevent exposure but not the loss of depth, which turns into harm when the harness is reused with a weaker model. GEPA kept every safety rule verbatim, yet two of its three final harnesses misused credentials under a weaker model. A statistical model frozen before the runs predicts where each gate's protection breaks as safeguard cost and batch size change, and freezing safety text or auditing one-layer ablations preserves depth.