Agent benchmark compression improves evaluation speed and accuracy
Beyond Outcomes: Dual-View Relational Learning for Efficient Agent Benchmarking
Computation and LanguageArtificial Intelligence
Summary
Testing how well AI agents perform on large sets of tasks can take a lot of time and effort. The authors found that looking at both the final scores and how agents behave during tasks helps pick a smaller but still powerful set of tests. Their method, DualViewEval, reduces the number of tasks needed to evaluate agents while keeping predictions accurate. This makes it easier and faster to compare different AI agents and understand their strengths and weaknesses.
What this means in practice
- •For ai development teams: Efficiently evaluate and compare different AI agents using a compact set of representative tasks, speeding up agent benchmarking workflows.
- •For automated testing teams: Reduce time and resources needed for large-scale agent testing by selecting fewer but more informative tasks based on both agent behavior and outcomes.
Authors
Xinshuai Guo, Junjie Wu, Dolly Deng, Yinghui Li, Hai-Tao Zheng, Suncong Zheng, Maxm Pan
Abstract
Agent benchmarks are substantially more costly to evaluate than conventional LLM benchmarks. Benchmark compression is therefore a natural solution, yet existing methods primarily model redundancy in task--model final-score distributions, which is important in agentic evaluation. To address this limitation, we analyze large-scale trajectories and identify six complementary process signals that are systematically associated with final agent performance. To disentangle agent performance redundancy from a complete perspective, we propose DualViewEval, an agent benchmark compression method that jointly exploits outcome and process relations to learn an exact-size miniset and predict the full-benchmark scores. Across five agent benchmarks and five representative baselines, DualViewEval achieves the best results in all datasets. With only 20 tasks, it achieves $24\times$--$40\times$ compression on APEX-Agents and BFCL, reducing mean absolute error (MAE) by $14.5\%$--$28.2\%$ over the strongest competitors while improving Kendall's $τ$ by up to $7.2\%$ relative to EssenceBench on SWE-bench Verified. The selected minisets further reveal capability differences among different agents, providing compact and diagnostic feedback for efficient agentic model development.