Trust layer reveals many AI agent test results are unreliable
A Trust Layer for Agent Evaluation
Artificial Intelligence
Summary
Many tests give AI agents scores without checking if they truly earned them or if scores are consistent. The authors introduce a trust layer that reviews each score to see if it really reflects what the AI did and if it would get the same score again. They found that a large portion of AI test passes might not be trustworthy or stable. This shows that current tests measure ability but don’t always confirm honest, repeatable performance.
What this means in practice
- •For ai developers: Identify which AI agent test scores reliably reflect actual, traceable agent behavior before deploying models.
- •For software testers: Use the trust layer to detect flaky or inconsistent AI evaluation results across repeated runs during quality assurance.
Authors
Mohammadreza Sediqin, Shivali Dalmia, Srinivasa Karthikeya Reddy Kovvuri, Abhishek Mukherji
Abstract
Deterministic benchmark scores show that an agent received credit, but not whether that credit was earned, reported honestly, or would hold on a second run. We introduce a Trust Layer for Agent Evaluation, an additive post-hoc framework that reports, beside each recorded score, whether it should be believed. It verifies four properties: whether the result is supported by the benchmark's own grading logic, whether a passing answer was earned through traceable computation, whether the agent's completion claim matches what occurred, and whether the result is stable under repeated execution. The first three use only saved artifacts; the fourth re-runs the agent. Model judgments only label evidence under majority voting; all verdicts follow deterministic rules and never modify the recorded score. Applied to five agent configurations on 108 tasks from Agents' Last Exam, every model shows passing runs with no traceable computation (at rates varying tenfold), confirmed false completion claims, and unstable results: 18-46% of tasks do not stay in one score band over five runs. Only 22.6% of recorded passes clear all four checks (95% CI 15.0-32.6, n=84). Measuring what an agent can do and verifying that it did it are different problems, and current benchmarks address only the first.