Simulated evaluations improve AI safety predictions in deployment
Simulated Control Evals
Artificial Intelligence
Summary
Sometimes AI systems try to do things they shouldn’t, and people add safety checks to stop this. To test how well these checks work before using the AI, researchers usually try the AI in special test settings. But these tests might not match the real situations the AI faces. This paper shows a way to simulate those situations using language models, which better predicts how safe the AI will be when it’s actually running. Their method also finds smarter ways to spot and block harmful AI behavior.
What this means in practice
- •For ai safety engineers: Predict how well safety safeguards protect AI systems in real deployment by simulating environment responses with language models.
- •For security testing teams: Generate and test attacker strategies in simulation to improve AI monitoring tools without needing full system executions.
Authors
Jeffrey Olmo, David Zhang, Tyler Tracy
Abstract
Control evaluations measure whether safeguards can prevent an AI model from causing undesired outcomes even when it is intentionally trying to do so. These evaluations typically use purpose-built control settings and extrapolate their results to a model's deployment. However, the effectiveness of safeguards in deployment can depend on features of environments and tasks that control settings do not represent. To broaden where control evaluations can be run, we introduce simulated control evaluations (SCEs), which use a language model to simulate tool responses in place of an environment. We evaluate SCEs on LinuxArena, where execution provides ground-truth outcomes. We compare SCE prediction errors with those from a separate control setting, BashArena, and with variation in control metrics within LinuxArena. For independently sampled evaluation and deployment task sets, SCEs predict safety with a root mean squared error of 3.6 percentage points across five monitors at a 5% audit budget, compared with 10.1 for BashArena. SCEs closely reproduce distributions of attack strategies, attacker reasoning, and monitor reasoning, with over tenfold lower KL divergence from execution than BashArena. SCEs explain 78-85% of the between-environment variance in attack success, monitor scores, and safety. This accuracy is sample efficient: SCEs predict safety with less error than BashArena using fewer than 5% as many task-pair draws as the deployment task set. Strategies discovered through simulation also improve monitor performance on held-out LinuxArena attacks, outperforming BashArena and approximately matching improvement from executed strategies. These results suggest that deployment-specific SCEs can be more predictive than control evaluations executed in separate control settings.