RADAR: Rubric-Aware Dependency and Redundancy Analysis for LLM-as-Judge Evaluation
2026-08-03 • Computation and Language
Computation and Language
AI summaryⓘ
The authors present RADAR, a tool that checks if different parts of a scoring rubric influence each other unexpectedly when evaluating AI models. Instead of assuming each evaluation criterion acts independently, RADAR tests small example cases to see how scores on one criterion might affect others. This helps identify if scores are redundant or linked before doing a big evaluation, saving effort and ensuring fairer assessments. The authors tested RADAR in several real-world settings and found it reliably matches how humans see the relationships between criteria.
Large Language Models (LLMs)rubric-based evaluationbehavioral couplingevaluation criteriasynthetic probescorrelation matrixmodel evaluationinter-criterion correlationbenchmarkingaudit framework
Authors
Divyansh Singh, Reza Davari, Afra Mashhadi
Abstract
Rubric-based LLM-as-judge pipelines often assume that evaluation criteria provide independent signals. In practice, however, criteria can be behaviorally coupled: improving one criterion may systematically change scores on another, distorting aggregate scores used in model-release or product-update decisions. We introduce RADAR, a lightweight preflight diagnostic framework for estimating such coupling before large-scale evaluation. Given a rubric, RADAR generates targeted synthetic probes, scores each probe on all criteria, and produces a directional coupling matrix that shows which criteria co-score and how. We validate RADAR on three industry-relevant evaluation settings: NVIDIA HelpSteer2, SumPubMed, and the Yale-Salesforce SummEval benchmark. Using only a small number of probes per criterion, RADAR recovers human inter-criterion correlation structure (Pearson r > 0.84) and provides practitioners with concrete audit signals about redundancy, hierarchy, and aggregation sensitivity before committing to large-scale judging.