SkillAdam improves training of language model agent skills with stable efficient updates
SkillAdam: Stable and Efficient Skill Evolution for Agents
Artificial Intelligence
Summary
Training language model agents to have useful skills can be slow and unstable because improving those skills often relies on trial-and-error updates that may undo earlier progress. Researchers present SkillAdam, a method inspired by the Adam optimization algorithm, designed to make skill improvements more stable and efficient. SkillAdam keeps track of past changes and how successful they were, then adjusts how much it edits skills based on recent results, preventing wild changes. This approach helps agents learn better skills using fewer updates and less cost, working well across different types of tasks.
language model agentsskillsskill evolutionoptimization algorithmsAdam optimizerupdate stabilityiteration efficiencynon-differentiable optimizationexecution feedback
Authors
Gaoyuan Li, Meihao Fan, Yizhe Liu, Shaolei Zhang, Ju Fan, Siyi Wang, Jiaheng Hou, Xudong Weng, Honghan Tian, Zang Li
Abstract
Agent skills provide a lightweight way to equip frozen language-model agents with domain knowledge and procedural guidance, yet obtaining high-quality skills remains costly and difficult to scale. Expert-written skills require substantial human effort. Recent skill self-evolution methods automate an iterative loop that uses execution feedback to revise skills, but their heuristic update strategies often yield unstable optimization and low iteration efficiency. We identify two challenges in realizing stable and efficient skill self-evolution. Direction Stability requires effective corrections to accumulate rather than be overwritten by iteration-local feedback. Update Adaptivity requires the scope of each revision to reflect the consistency of recent case-level improvements. We introduce SkillAdam, an Adam-inspired framework for optimizing discrete and non-differentiable skill documents. As a functional analogue of Adam's first moment, an optimization memory records identified problems and the outcomes of prior solution attempts to stabilize the update direction. As a functional analogue of Adam's second moment, a volatility-driven edit budget tracks the history-weighted variation of recent case-level improvements and adaptively controls the update magnitude. Across seven benchmarks that span short- and long-horizon tasks, SkillAdam achieves state-of-the-art performance with more stable optimization dynamics. It also obtains stronger skills with substantially fewer optimization iterations and lower cost than prior methods. Code repository: https://github.com/ruc-datalab/SkillAdam