Cargo improves ai evaluation by using context to avoid false errors
CARGO: Context-Aware Retrieval-Gated Evaluation of Agentic AI in Production
Computation and LanguageArtificial IntelligenceMachine Learning
Summary
Evaluating AI systems that handle live data like support cases can be tricky because the reference answers usually refer to different cases with different details. The authors show that traditional evaluation methods wrongly mark correct answers as mistakes due to these differences. They propose CARGO, a way to judge AI outputs by considering the actual context and only penalizing true contradictions. Their tests show CARGO avoids false penalties while still catching real errors, although it struggles with some types of mistakes related to procedure.
What this means in practice
- •For ai ops teams: Evaluate deployed AI agents that operate over live entities more accurately by reducing false error penalties caused by entity differences.
- •For customer support developers: Improve testing of AI systems handling support cases by grounding evaluations in each live case's data to better identify true procedural errors.
Authors
Mukul Chhabra, Shail Patel, Luigi Medrano
Abstract
Reference-based LLM-as-a-judge evaluation assumes the reference answer is the target. In deployed agentic systems that operate over dynamic entities (support cases, assets, accounts), the closest available reference typically applies the correct procedure to a different entity, so a literal judge penalizes different identifiers, dates, and statuses as errors or hallucinations. We name this failure mode reference-instance divergence (RID). We propose CARGO, a framework that (i) treats retrieved references as procedural exemplars and grounds factual judgments in the live instance's observed context, (ii) assigns each claim a three-way status (supported, contradicted, unverifiable) and penalizes only contradictions, and (iii) gates evaluation by retrieval confidence, casting production evaluation as selective prediction. We introduce CARGO-Bench, a perturbation-based diagnostic suite with ground truth by construction that separates leniency from discrimination. On CARGO-Bench (246 items, two judge models, 7,872 judgments), the standard reference-based judge penalizes 100% of correct entity-transplanted answers and is uninformative (discrimination index DI ~ 0); supplying the live facts without reframing changes nothing. CARGO eliminates these false penalties (0/50) while retaining near-complete contradiction recall (50/50 and 49/50), raising DI to 0.58 [0.48, 0.68]; a rubric-swap control attributes most of the effect to context-grounded dimension definitions. CARGO also exposes a limitation of its own design: the leniency that protects entity values suppresses detection of procedural corruptions (20% recall). A post-hoc fix does not close the gap, and an LLM-as-annotator study with written guidelines and adjudication shows the same blind spot. We release a preregistered protocol for extending the evaluation to expert agreement, risk-coverage, and cost on production traffic.