Installed coding skills often conflict causing hidden loss of key functions
One Skill Too Many: How Co-Installed Skills Conflict in Coding Agents
Software EngineeringArtificial Intelligence
Summary
Coding assistants use special skills to help with tasks, but sometimes two similar skills get installed together and interfere with each other. This causes one skill to run instead of the other, which can silently remove important functions even while the task still completes successfully. The authors studied thousands of code repositories and confirmed that these conflicts are common and hard to detect. They suggest new ways to test and display which skill actually runs to avoid hidden problems.
What this means in practice
- •For software platform developers: Develop platforms that detect and display which skill runs first to preserve exclusive functions and prevent silent skill conflicts in coding assistants.
- •For software quality engineers: Create benchmarks that measure exclusive core functions to catch cases where tasks pass but key skill functions are lost due to conflicts.
Authors
Chaoliang Yan, Zihao Xu, Yuekang Li, Shangzhi Xu, Yi Liu, Gelei Deng, Siqi Ma
Abstract
Coding agents are extended with agent skills, directories whose SKILL.md tells the model when and how to perform a task. Because skills come from independent sources (teams, developers, plugins, copied collections), an installed skill can be co-installed with a similar skill doing the same job, and the model picks between them by name and description alone. In a conflict, the installed skill loses core functions (e.g., a ban on touching git) because the similar skill runs instead or changes what it does. The task still passes, so benchmarks that check only task completion miss such cases. We present the first empirical study of such conflicts. From snapshots of 20,947 repositories, we mine 822,109 candidate similar-skill pairs, have an LLM judge a stratified sample of 3,754, and run 312 confirmed pairs on three models (6,368 runs, 169,294 tool calls, 542 agent-hours). We report five findings. (1) Conflict-prone skills are common: nearly one in four installed skills is co-installed with one that does the same job, and 37% of judged skills sit inside copied collections. (2) Most such pairs involve normative skills, then capability skills. (3) Without lowering task completion, a similar skill takes one in five runs from the installed skill, and runs that open the similar skill first lose over a third of the exclusive core functions that only the installed skill fulfills. (4) Install location decides which skill runs, listing order barely matters, and the final reply names the skill used in only 0.9% of substituted runs. (5) Conflicts are decided at the first skill read, almost always before any file is changed, and a pre-tool hook at that read restores fidelity on exclusive core functions to the level of runs that open the installed skill first. Benchmarks should thus score exclusive core functions, and platforms should guard the first read and show which skill ran.