SkillAdam improves agent skill learning with stable and fast updates
SkillAdam: Stable and Efficient Skill Evolution for Agents
Artificial Intelligence
Summary
Teaching AI agents new skills can be slow and unstable because updating these skills usually relies on guesswork. The authors found two key problems: updates can undo previous progress and sometimes revisions are too big or too small. They created SkillAdam, a method inspired by a popular optimization algorithm, to keep track of past updates and adjust how much skills change based on recent improvements. This makes the learning process steadier and faster across many types of tasks.
What this means in practice
- •For language model developers: Improve the efficiency and stability of iterative skill refinement in pretrained AI agents using discrete feedback.
- •For automation engineers: Enhance domain-specific task automation by stabilizing skill updates in AI systems that operate with fixed foundational models.
Authors
Gaoyuan Li, Meihao Fan, Yizhe Liu, Shaolei Zhang, Ju Fan, Siyi Wang, Jiaheng Hou, Xudong Weng, Honghan Tian, Zang Li
Abstract
Agent skills provide a lightweight way to equip frozen language-model agents with domain knowledge and procedural guidance, yet obtaining high-quality skills remains costly and difficult to scale. Expert-written skills require substantial human effort. Recent skill self-evolution methods automate an iterative loop that uses execution feedback to revise skills, but their heuristic update strategies often yield unstable optimization and low iteration efficiency. We identify two challenges in realizing stable and efficient skill self-evolution. Direction Stability requires effective corrections to accumulate rather than be overwritten by iteration-local feedback. Update Adaptivity requires the scope of each revision to reflect the consistency of recent case-level improvements. We introduce SkillAdam, an Adam-inspired framework for optimizing discrete and non-differentiable skill documents. As a functional analogue of Adam's first moment, an optimization memory records identified problems and the outcomes of prior solution attempts to stabilize the update direction. As a functional analogue of Adam's second moment, a volatility-driven edit budget tracks the history-weighted variation of recent case-level improvements and adaptively controls the update magnitude. Across seven benchmarks that span short- and long-horizon tasks, SkillAdam achieves state-of-the-art performance with more stable optimization dynamics. It also obtains stronger skills with substantially fewer optimization iterations and lower cost than prior methods. Code repository: https://github.com/ruc-datalab/SkillAdam