Rubric evaluations get more accurate by weighting uncertain judgments
Rubric Judgments Lack Uncertainty: Conformal Weighting for LLM Evaluation
Artificial Intelligence
Summary
Rubric-based reviews of AI text often treat all parts of the evaluation as equally clear and certain, but some judgments are more uncertain than others. The authors propose a way to measure how uncertain each part is by using similar question versions and a math method called conformal calibration. This approach gives different weights to each criterion based on certainty, improving the overall evaluation and helping AI be trained better. Their tests show more accurate preference decisions and improved AI alignment when this method is used.
What this means in practice
- •For machine learning engineers: Improve AI training by refining how reward signals weight uncertain rubric judgments in reinforcement learning.
- •For ai quality assurance teams: Increase accuracy in AI output evaluation by accounting for uncertainty in rubric-based assessments.
Authors
Licheng Pan, Haocheng Yang, Haoxuan Li, Yuzhou Chen, Yichen Sun, Yuan Lu, Xiaoxi Li, Hao Wang, Zhixuan Chu, Kui Ren
Abstract
Rubric-based evaluation is widely used to assess large language model (LLM) outputs by decomposing quality into multiple criteria and assigning each a binary judgment from an LLM verifier. However, judgment certainty varies substantially across criteria. Existing methods uniformly aggregate these judgments into a final score, implicitly treating every criterion as equally certain and overlooking this heterogeneity. As a result, uncertain judgments remain incorporated into reward estimation without distinction, potentially yielding misleading training signals for downstream reinforcement learning (RL). To address this limitation, we propose Uncertainty-aware Rubric Weighting (URW), a training-free framework that quantifies criterion-level judgment uncertainty via conformal calibration. Specifically, URW elicits a soft uncertainty signal from a fixed pool of paraphrased judgment templates and converts it into a per-criterion weight using split-conformal $p$-values with provable coverage guarantees. Extensive experiments demonstrate that URW consistently improves pairwise preference accuracy and delivers stronger downstream RL alignment.