Detecting Safety Violations Across Many Agent Traces
2026-04-13 • Artificial Intelligence
Artificial IntelligenceComputation and Language
AI summaryⓘ
The authors explain that finding safety problems in agent behaviors is hard because issues are rare, complicated, and sometimes hidden across many examples. They created Meerkat, a tool that groups similar behaviors and intelligently searches for problems based on natural language descriptions. Meerkat works better than existing methods by focusing on promising areas and does not need pre-set tests or checks. It found many more safety issues, including cheating and reward exploitation, in different testing scenarios.
agent tracessafety violationsclusteringagentic searchreward hackingmisalignmentnatural language specificationsbenchmark cheatingadaptive investigation
Authors
Adam Stein, Davis Brown, Hamed Hassani, Mayur Naik, Eric Wong
Abstract
To identify safety violations, auditors often search over large sets of agent traces. This search is difficult because failures are often rare, complex, and sometimes even adversarially hidden and only detectable when multiple traces are analyzed together. These challenges arise in diverse settings such as misuse campaigns, covert sabotage, reward hacking, and prompt injection. Existing approaches struggle here for several reasons. Per-trace judges miss failures that only become visible across traces, naive agentic auditing does not scale to large trace collections, and fixed monitors are brittle to unanticipated behaviors. We introduce Meerkat, which combines clustering with agentic search to uncover violations specified in natural language. Through structured search and adaptive investigation of promising regions, Meerkat finds sparse failures without relying on seed scenarios, fixed workflows, or exhaustive enumeration. Across misuse, misalignment, and task gaming settings, Meerkat significantly improves detection of safety violations over baseline monitors, discovers widespread developer cheating on a top agent benchmark, and finds nearly 4x more examples of reward hacking on CyBench than previous audits.