Papers for

ai audit companies

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Auditors face difficulty detecting sabotage in ai safety research logs

Auditors Struggle to Catch Sabotage In End-to-End Transcripts of AI Safety Research

Abstract: As AI systems are increasingly used to automate AI R&D, misaligned systems could sabotage research intended to improve model safety and alignment. To stress-test defenses against such sabotage, we use a human expert as a proxy saboteur on AI safety research. We conduct a red-team/blue-team game where a human researcher runs AI safety experiments from scratch and auditors try to tell which were sabotaged and how. Given access to the full research transcript and the ability to run experiments, human auditors mostly fail to distinguish sabotaged from benign research: they achieve an AUROC of 0.59, not significantly better than chance, and fully identify the specific sabotage in only 17% of cases. Despite not being allowed to run experiments, the best frontier models outperform human auditors (Fable 5.1: AUROC 0.73, 32% full identification) within noise and weaker models perform comparably. When the saboteur chooses which experiments are attacked, rather than being required to sabotage every question given, sabotage identification drops further. Our results suggest that research sabotage by expert-level agents is difficult to detect with current auditing methods and provide a new setting for studying research sabotage.

Mon 16 NovArtificial Intelligence
The gist
The paper shows that people trying to find deliberate mistakes or sabotage in detailed records of AI safety research mostly fail to spot them. Even when given complete research notes and allowed to rerun experiments, human auditors performed little better than guessing. Interestingly, some advanced AI models did better at spotting sabotage than humans. The study highlights the challenge of catching sabotage by experts working to improve AI safety using current checking methods.
Open → 2611.21832v1