Evaluating benchmarks reveals varied reliability across language tasks
Are Benchmarks Reliable? Toward Structural Diagnosis via Sample-Level Capability Boundaries
Artificial Intelligence
Summary
Benchmarks are used to judge how well language AI models perform, but overall scores can hide problems in how well individual test questions compare models. The authors developed BSDProbe, a tool that looks closely at each question’s difficulty and consistency to see if benchmarks truly show differences between models. They found some benchmarks are more reliable than others and that examining individual questions gives a better idea of how trustworthy comparisons are. Their method also helps pick smaller sets of questions that can compare models much more effectively.
What this means in practice
- •For machine learning engineers: Improve evaluation by selecting smaller, more reliable subsets of benchmark questions to compare language models with higher confidence.
- •For ai product managers: Assess and communicate the trustworthiness of benchmark scores when choosing language models for deployment in real products.
Authors
Haiquan Hu, Yuzhu Liang, Weicheng Tang, Yanzeng Li, Yao Shi, Tian Wang
Abstract
Evaluating large language models (LLMs) relies heavily on benchmark scores, yet aggregate metrics can obscure whether benchmark samples reliably support model comparison. We introduce \textbf{BSDProbe}, a sample-level framework for \emph{benchmark structural diagnosis} that estimates capability boundaries from repeated-response trajectories along ordered model axes. BSDProbe summarizes samples by boundary position, boundary width, boundary-signal validity, and order consistency, then aggregates them into benchmark-level structural profiles. Experiments on six benchmarks show that benchmark reliability is axis-conditioned and heterogeneous: GSM8K and MATH exhibit the most stable measurement structures, MMLU and TriviaQA are relatively stable but heterogeneous, while GPQA and PopQA show stronger axis-conditioned risks. These profiles remain consistent across Qwen3, Qwen2.5, and cross-model axes. BSDProbe further selects compact high-value subsets whose model discriminability reaches up to $8.58\times$ that of the full benchmark. These results suggest that reliable benchmark use requires examining sample-level capability boundaries beyond leaderboard scores.