Skill reranking improves with difference-focused text selection
SkillContrast: Difference-Guided Text Selection for Agent Skill Reranking
Information RetrievalArtificial Intelligence
Summary
Finding the right skill for a task can be tricky when many skills share similar instructions but differ in key details. The authors created SkillContrast, a way to pick out just the parts that are different between similar skills so a computer can better rank which skill fits a request. This method helps computers find more useful skills without mixing up risky ones, and it does this while using less text. By focusing on differences rather than all the shared information, SkillContrast improves the precision of skill selection.
What this means in practice
- •For ai chatbot developers: Improve chatbot skill selection by focusing on differences between similar instructions to provide safer and more relevant responses.
- •For customer support automation teams: Enhance automated support systems to avoid risky suggestions by better distinguishing which skill fits a specific customer query.
Authors
Jiandong Ding, Honglei Ji, Ming Liu, Tao Duan
Abstract
Similar agent skills can share instructions but differ in their conditions of use. Query-based text selection may retain shared instructions and omit these distinctions. We introduce SkillContrast, a training-free selector that compares retrieved skills and retains their differing text with local context for a pretrained reranker. On 1,235 requests from SameCapRisk-Bench, it yields 54-72 more clean hits (requests that retrieve a helpful skill without its marked risky sibling) than TF-IDF query selection at identical per-candidate input lengths, across 2 retrievers and 2 reranker sizes. Length-matched component replacements identify differing text as the main contributor in the primary setting, with smaller, mixed context effects. Relative to full skill bodies, SkillContrast uses 51.1-58.8% fewer model-input tokens, with 10-18 fewer clean hits at 0.6B and matching or higher observed clean-hit counts at 4B. Candidate-relative differences thus complement query relevance in selecting compact reranking inputs.