Zep agent skills improve AI task success by learning from verified attempts

Learning by Doing: Autonomously Compiling and Curating Agent Skills from Verified Traces

Artificial IntelligenceComputation and Language

Summary

Many AI systems remember facts or past conversations but struggle to learn how to perform tasks on their own. The authors describe Zep Agent Skills, a system that watches verified attempts by stronger AI models completing tasks, then turns those attempts into reusable skill sets. These skills help weaker AI models do better on the same tasks without needing further training. This approach also organizes and ranks the skills for easy use by different AI agents.

What this means in practice

  • For enterprise ai teams: Integrate verified procedural skills to boost AI agents’ performance on business process tasks without retraining.
  • For software developers: Access a standardized skill format to assemble and reuse AI task procedures across different agent platforms.
  • For customer service automation teams: Deploy skills learned from expert AI interactions to improve automated handling of banking and operational queries.$Commercial implications: Enables selling enhanced AI customer support tools that improve accuracy by using skill sets derived from verified agent runs.

Authors

Daniel Chalef, Peter Evans, Pavlo Paliychuk, Mehul Patel, Preston Rasmussen, Jack Ryan

Abstract

Memory systems for production LLM agents mostly store semantic memory (facts about users and domains) or episodic memory (earlier interactions). Research systems such as Trace2Skill, WikiSkill, and SkillRevise learn procedures from agent runs, package them as Skills, and evaluate them on benchmarks. We describe Zep Agent Skills, a production service that gives deployed agents this procedural memory. The service uses only verified attempts as evidence, cites the supporting events for each field of a Skill, and governs the admission and distribution of each Skill version. The service records each task attempt as a Trace with ordered events, an outcome, and a verification strength. A compiler with separate Reflector and Curator roles turns verified Traces into versioned Skills. Scans and admission gates control which candidates become active, and verified use outcomes change the search rank of each Skill. The Agent Skills format is an open standard that Anthropic originated and donated to the Agentic AI Foundation, so any host of the format can load a compiled Skill. Agents retrieve Skills through lexical search with progressive disclosure, over an API or MCP. We evaluate Zep Agent Skills on $τ^3$-bench banking, AppWorld, and EnterpriseOps-Gym. Skills compiled from the Traces of a teacher model increase the pass rate of three weaker executor models on the same tasks, at every reasoning effort, by 4.5 to 18.8 points. Most of the gain is on the tasks that the teacher solved. On ALFWorld, Skills compiled from the runs of a small model give a gain similar to or larger than the gain that prior work reports for continuous fine-tuning, with no training.