CodeSkill improves long-term code generation with skill abstractions
CodeSkill: Latent Skill Abstraction for Long-Horizon Code Agents
Artificial Intelligence
Summary
Making computer programs by teaching AI is hard because the AI has to think many steps ahead and the small pieces of code it writes may not directly show success or failure. The authors developed CodeSkill, a new way that helps AI break down coding tasks into bigger, meaningful skills rather than focusing on tiny code parts. This approach helps the AI learn more efficiently and perform better on long, complex coding problems. The authors tested it on many coding challenges and showed that it works well and can be used in different coding areas.
What this means in practice
- •For software development teams: Use CodeSkill to train AI coding assistants that manage complex, multi-step programming tasks more efficiently with better long-term reasoning.
- •For ai platform engineers: Incorporate hierarchical skill abstractions into frozen LLMs to optimize code generation policies in a compact semantic space rather than token by token.
Authors
Song-Li Wu, Jingyi Wang, Zhaocheng Du, Weinan Gan, Weiwen Liu
Abstract
Code agents require long-horizon decision-making over complex interaction trajectories. However, existing reinforcement learning (RL) approaches typically optimize behavior at the token level, creating a mismatch between low-level generation and high-level behavioral reasoning. This limitation leads to inefficient exploration and weak credit assignment under sparse rewards. Moreover, while large-scale agent trajectories often contain recurring multi-step behavioral patterns, their noisy token-level representations hinder effective experience reuse. To address these challenges, we propose CodeSkill, a framework that adapts hierarchical latent skill modeling to the code agent domain. CodeSkill first leverages a teacher model to distill both successful and failed trajectories into multi-level textual abstractions. It then integrates temporal variational inference with reinforcement learning to map these discrete semantics into continuous latent variables, while an adaptive boundary mechanism dynamically gates skill transitions based on execution feedback. The learned skills are injected into a frozen LLM policy as latent semantic prefixes, enabling optimization in a compact semantic space rather than over raw token sequences. By shifting RL from token-level exploration to experience-level reasoning, CodeSkill improves optimization efficiency and long-horizon behavioral coherence. Extensive experiments demonstrate that CodeSkill achieves highly competitive performance against strong open-weight baselines across diverse general and industrial coding benchmarks. Furthermore, the learned skills exhibit strong transferability and robust cross-domain generalization, highlighting the effectiveness of explicit behavioral abstraction for scalable agentic code generation.