LLM agents improve skills by analyzing their own thought patterns
Rep2Skill: Representation-Guided Skill Self-Evolution for LLM Agents
Artificial Intelligence
Summary
Large language model (LLM) agents learn reusable skills to complete tasks without changing their underlying code. However, previous methods improved these skills only by looking at the final text outputs and task results, missing the rich internal thought processes the agent went through. The authors introduce Rep2Skill, a way for agents to look inside their own internal decision-making steps to spot where things went wrong and fix their skills with targeted feedback. Tests show this method helps agents get better faster than just using text feedback alone. This work moves AI agents closer to improving themselves more like humans do, by reflecting on their own thinking.
What this means in practice
- •For chatbot developers: Upgrade chatbot abilities by enabling self-correction based on their internal reasoning paths rather than relying only on final outputs.
- •For robot control engineers: Improve autonomous robot task execution by allowing controllers to refine skills through analysis of their internal state changes during tasks.
Authors
Kaixing Zhang, Changming Li, Yingdong Shi, Zheng Zhang, Kaitao Song, Wenjie Shi, Jingang Wang, Kan Ren
Abstract
Textual skills enable large language model (LLM) based agents to accumulate reusable procedural knowledge without updating model parameters. Yet existing skill evolution remains largely confined to the text space: an optimizer must diagnose success and failure patterns, and revise skills solely from long execution trajectories and sparse task outcomes. This text-only paradigm leaves the agent's internal representations, which contain rich records of its evolving execution state, outside the skill optimization loop. We ask whether an agent can improve its external textual skills by reflecting on its own internal representations. We introduce Rep2Skill, a representation-guided framework for self-evolution on agent skills. Specifically, upon the collected agent rollouts, Rep2Skill models their internal model representation trajectories to localize turns that deviate from successful execution dynamics, and it further interprets these signals alongside the execution contexts as actionable textual feedback for targeted skill revision. Experiments on two agent environments with two open-source LLMs show that Rep2Skill consistently outperforms text-only approaches in the self-evolution setting, where the same LLM serves as both executor and optimizer without a stronger external model. This establishes a promising direction moving agent self-improvement beyond text-only reflection.