Framework improves evaluation and selection of AI agent skills
MCRI: A Four-Dimensional Framework for Analyzing and Evaluating Agent Skills
Artificial IntelligenceSoftware Engineering
Summary
AI agents often use different skills to perform tasks, but it’s hard to evaluate which skills work best. The authors introduce a new framework called MCRI to analyze and score these skills before running costly tests. They tested their approach on tens of thousands of skills and found that their method predicts skill usefulness better than others. This helps choose the best skills quickly and saves time and computing resources.
What this means in practice
- •For ai software developers: Improve AI agent design by quickly identifying the most promising skills to use in applications.
- •For cloud service operators: Reduce costs by prioritizing which AI skills to execute based on predictive evaluation before deployment.
Authors
Zongrui Yang, Li Xintong, Runchen Xu, Zhongsheng Wang, Zhedong Lin, Haoyuan Li, Jiamou Liu
Abstract
As agents evolve from single-tool systems into modular, composite architectures, skills are becoming an important mechanism for capability development and distribution. However, the academic community lacks a structured framework for systematically analyzing and evaluating skills. Drawing on information gain and behavioral constraint, we propose the four-dimensional MCRI Framework and operationalize it as MCRI-Eval, a large language model-based evaluation method. We evaluate MCRI-Eval using 63,812 public skills from the OpenClaw skill Hub, with 58,275 skill-conditioned model executions across BigCodeBench, BFCL-Fundamental, and Mind2Web. MCRI-Eval scores are positively associated with community popularity signals and achieve the highest downstream ranking agreement among the evaluated methods. MCRI-Eval also improves top-1 skill selection across all three benchmarks: compared with the strongest baseline on each benchmark, the skills selected by MCRI-Eval advance by 17.7, 22.8, and 19.6 percentile points in downstream performance rank on BigCodeBench, BFCL-Fundamental, and Mind2Web, respectively. These results indicate that MCRI-Eval provides a useful pre-execution signal for prioritizing promising skills before costly execution-based evaluation.