Auditors face difficulty detecting sabotage in ai safety research logs

Auditors Struggle to Catch Sabotage In End-to-End Transcripts of AI Safety Research

Artificial Intelligence

Summary

The paper shows that people trying to find deliberate mistakes or sabotage in detailed records of AI safety research mostly fail to spot them. Even when given complete research notes and allowed to rerun experiments, human auditors performed little better than guessing. Interestingly, some advanced AI models did better at spotting sabotage than humans. The study highlights the challenge of catching sabotage by experts working to improve AI safety using current checking methods.

What this means in practice

  • For ai research teams: Improve workflows to monitor potential sabotage in AI safety experiments using transcript analysis.
  • For ai audit companies: Develop better tools to detect sabotage within scientific research documents using ML models based on this study’s findings.$Commercial implications: Enables creation of advanced AI-driven research auditing software to sell to organizations needing reliable verification.

Authors

Keshav Shenoy, Anthony Zang, Eric Gan, Jacqueline Tay, Amy Ngo, Sebastian Prasanna, Julian Stastny, Vivek Hebbar

Abstract

As AI systems are increasingly used to automate AI R&D, misaligned systems could sabotage research intended to improve model safety and alignment. To stress-test defenses against such sabotage, we use a human expert as a proxy saboteur on AI safety research. We conduct a red-team/blue-team game where a human researcher runs AI safety experiments from scratch and auditors try to tell which were sabotaged and how. Given access to the full research transcript and the ability to run experiments, human auditors mostly fail to distinguish sabotaged from benign research: they achieve an AUROC of 0.59, not significantly better than chance, and fully identify the specific sabotage in only 17% of cases. Despite not being allowed to run experiments, the best frontier models outperform human auditors (Fable 5.1: AUROC 0.73, 32% full identification) within noise and weaker models perform comparably. When the saboteur chooses which experiments are attacked, rather than being required to sabotage every question given, sabotage identification drops further. Our results suggest that research sabotage by expert-level agents is difficult to detect with current auditing methods and provide a new setting for studying research sabotage.