Prompt2Skill improves task solving by building skills from instructions
Prompt2Skill: Unsupervised Skill Optimization From Natural Language Instructions
Computation and LanguageArtificial Intelligence
Summary
Many AI models rely on special skills, which are like step-by-step guides, to do tricky tasks better. Writing these skills by hand is costly and often not perfectly matched to each AI model. The authors created Prompt2Skill, a way for AI to build and improve these skills all by itself from just a description of the task. It works better than just giving the AI direct instructions, across different kinds of tasks like math or answering questions.
What this means in practice
- •For ai application developers: Generate and refine custom task skills for language models from user instructions without needing curated training data.
- •For business process automation teams: Create adaptive AI skills for specialized workflows to improve handling of unique tasks in domains like spreadsheets or document analysis.
Authors
Bo Ni, Li Li, Ryan A. Rossi, Franck Dernoncourt, Tyler Derr
Abstract
Skills are external artifacts that Large Language Models (LLMs) consume at inference time to improve their performance on specialized domains by incorporating relevant procedural and domain knowledge. Expert-authored skills are expensive to produce, and the resulting artifacts are not optimized for the specific model that consumes them, whose failure modes can vary with version, scale and training. In addition, emerging tasks may fall outside the scope of existing skill libraries, creating a need to develop new skills before curated training data become available. Recent works have explored automated skill optimization through reflection, but they require a curated, in-distribution training set, which users might not always have. To address these limitations, we present Prompt2Skill, a framework that builds skills from natural-language task description alone. From the prompt, the system derives a task specification, discovers or synthesizes datasets, and refines the skill in a closed loop of reflective editing. Across four domains spanning question answering, reading comprehension, spreadsheet manipulation, and mathematical reasoning, Prompt2Skill consistently outperforms the direct prompting baseline, achieving an average improvement of 10.8 across open-source and frontier models.