Stronger judges accept more manipulated outputs in AI evaluation

Stronger Judges Do Not See More: Deployment-Integrity Blindness in LLM Evaluation

Cryptography and SecurityArtificial Intelligence

Summary

Evaluating large language models (LLMs) often means checking only their final answers, not how they run behind the scenes. The authors found that even very advanced evaluators can’t spot when the system is secretly tampered with, as long as the outputs look normal. They created a test benchmark to measure how often these tampered outputs slip through undetected, and interestingly, better evaluators were more likely to accept them. They also showed that some hidden changes are impossible to catch by just checking answers, but verifying the system’s identity directly can detect them right away.

What this means in practice

Authors

Jianchang Su, Dawei Xiang, Yifan Zhang, Wei Zhang

Abstract

LLM-agent evaluation reports behavioral scores but leaves the identity of the deployed runtime stack unreported. A tampered deployment that preserves evaluated outputs passes every output-only evaluator. We introduce DIB-Bench, a benchmark that measures this deployment-integrity blindness across 7,560 scored examples, six tamper types, and an evaluator ladder from regex scorers through LLM judges to GPT-5-nano. Stronger evaluators accept more tampered outputs: mean acceptance rises from 88.6% (rule-based) to 94.1% (GPT-5-nano), and a trigger-conditional tamper invisible on the evaluation suite produces 48-100% harmful outputs, rising with model capability. A PAC-style bound shows that tampers with zero output shift are undetectable from outputs at any sample size; a manifest-hash identity channel detects them with zero samples. Behavioral fingerprinting identifies the model component above chance (76.7%) but covers one of six stack components. We also propose a graded deployment-identity descriptor for evaluation reports.