Skill comparison improves agent task success using capability graphs
SKALOS: Learning to Distinguish Similar Skills with Capability Graphs
Artificial Intelligence
Summary
Large language models can use various skills to help agents perform tasks, but it’s hard to pick the right skill when many look similar. The authors created SKALOS, a system that compares skills by looking at detailed capability information organized as graphs instead of just text descriptions. This helps distinguish subtle differences and choose the best skill, improving task success rates. They tested SKALOS on benchmarks showing better ranking of skills and more successful agent task completions.
What this means in practice
- •For agent developers: Enhance software agents by improving selection among similar skills to increase task success rates in complex environments.
- •For automation engineers: Improve robotic or software automation by distinguishing fine-grained procedural capabilities for better task planning.
Authors
Wenhan Xiao, Ziwei Zhang, Xingcheng Fu, Chuanyue Yu, Haonan Yuan, Qingyun Sun, Runhua Xu, Jianxin Li
Abstract
Skills provide LLM agents with reusable procedural knowledge, making appropriate skill selection essential for reliable agent behavior. As skill libraries expand, distinguishing semantically similar candidates becomes increasingly difficult. Existing methods encode or rewrite skill descriptions for relevance-based ranking, yet heterogeneous descriptions can obscure capability differences. To address this challenge, we propose SKALOS, a graph-native reranker that compares skills via explicit capability evidence. Its multi-hierarchy graph aligns equivalent expressions across skills while preserving skill-specific capability evidence. Since individually relevant capability elements may not jointly support a query, semiring-based capability witness learning scores each branch by its weakest evidence, checks joint support for the extracted requirements within one implementation, and selects the strongest alternative. To reduce the influence of extraction errors, we develop ranking-feedback graph calibration that adjusts unreliable evidence for candidate comparison. Experiments on SkillRet show that SKALOS improves NDCG@10 by 2.68 percentage points over a matched text-only reranker. Witness-guided skill selection also increases ALFWorld task success rate from 22% to 29%.