Code skill server boosts code generation quality across models
Code Skill Server: A Large-Scale Search Engine for Code Generation Skills
Software EngineeringInformation Retrieval
Summary
Finding the right pieces of code skills to help with coding tasks is getting too hard because there are so many of them out there. The authors created a tool called the Code Skill Server (CSS) that gathers and organizes almost a thousand useful code skills from different places. Their experiments show that using CSS helps code-generating AI models write better code, improving success rates by up to nearly 14%. This tool makes it easier and faster to find the right skills to help AI write code.
What this means in practice
- •For software development teams: Access a large searchable repository to find code generation skills that improve AI-assisted programming tasks.
- •For automated code generation tool developers: Integrate a scalable skill search engine to enhance code generation models with relevant coding skills from diverse sources.
Authors
Xiaoyong Li, Haolong Li, Dongliang Xu, Haoyu Li, Yifan Wang, Jingyuan Shi, Yanghua Wang, Yanbin Sun, Shidong Pan, Yue Yao
Abstract
Agent skills have proven to be an effective technique for improving agent ability on specific tasks, including software engineering, natural science, and mathematics. During such a process, using agent skills often involves manually discovering and organizing skills from heterogeneous sources, while existing skill retrieval methods mainly focus on selecting relevant skills from established skill libraries. However, with the rapidly increasing number and variety of reusable agent skills, manually searching for relevant skills across heterogeneous sources is becoming impractical at scale. In this paper, we introduce the Code Skill Server (CSS), a large-scale search engine for finding the most useful code generation skills for a target task. CSS consists of a skill server that incrementally collects and indexes publicly available code generation skills from multiple sources, with 993 skills included in the corpus used in this work. Extensive experiments across different retrieval methods and code generation benchmarks show that the best CSS configuration improves Pass@1 by up to 13.91% and consistently improves all evaluated models by at least 4.21% in the cross-model evaluation. These results demonstrate that CSS can effectively search for and serve relevant skills, improving code generation quality and enhancing models' code generation capabilities. Code and the CSS-993 corpus are available at {https://github.com/haoyuli-sdu/CodeSkillServer}.