SkillAlchemy: Open-World Agent Skill Creation
2026-08-24 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors explore how to create specialized agent skills when little information about the task is given, using materials from open sources instead of relying on human-written examples or past data. They introduce SkillAlchemy, a method that figures out hidden task requirements and chooses proper procedures based on evidence, then organizes these into a usable skill. Their tests show SkillAlchemy performs much better than not using skills at all and better than other automated methods, matching the quality of human-made skills. This means agents can learn new skills more reliably even when details are missing.
agent skillslanguage agentsskill creationopen-world learningSkillAlchemyprocedural artifactsinference timeSkillsBenchautomated skill synthesis
Authors
Hengjun Wang, Shuyue Wei, Boyi Liu, Jun Yang, Yongxin Tong
Abstract
Agent skills are reusable procedural artifacts that extend language agents with specialized workflows, tool conventions, and domain behaviors at inference time. However, creating reliable skills still depends largely on human authorship, model priors, or execution traces. These sources are often unavailable for unfamiliar tasks, suggesting the need to create skills from open-world materials. In this paper, we study open-world skill creation: given an underspecified skill brief and a source-access specification, a creator must discover behavior-relevant requirements omitted by the brief and determine how broadly each source-derived procedure is justified. We propose SkillAlchemy, an admission-centered framework for source-grounded skill creation. SkillAlchemy identifies implicit requirements through contrastive evidence, admits candidate procedures based on evidence-supported scope, and compiles the admitted content into a grammar-guided skill package. Extensive experiments across 87 SkillsBench v1.1 tasks demonstrate that SkillAlchemy improves pass rate over no-skill execution by 19.9 percentage points and the strongest automated baseline by 8.6 percentage points, while achieving performance comparable to human-curated skills.