Knowledge guided evolution improves kernels for Ascend NPUs
Beyond Experience: Knowledge-Guided Evolution for NPU Kernel Generation
Artificial Intelligence
Summary
Computers that run special tasks often need tiny programs called kernels, but making these kernels for new hardware is tough. The authors found that simply learning from past attempts isn’t enough because new situations need new ideas. They built a tool called ASCENDEVOLVE that uses a big collection of expert knowledge to guide the search for better kernels. This method helps it try smarter options, fix problems, and combine good ideas, leading to better results on a challenging chip called Ascend NPU.
What this means in practice
- •For embedded system developers: Generate efficient compute kernels for Ascend NPUs by integrating documentation and code knowledge into evolutionary search.
- •For hardware acceleration engineers: Improve operator implementations on emerging AI accelerators using knowledge-guided evolutionary methods to expand kernel optimization strategies.
Authors
Xi Chen, Ming Shen, Haotian Yan, Ruoyu Sun, Tianjun Yao
Abstract
LLM agents have become capable kernel engineers on mainstream GPU stacks, but they struggle on emerging accelerators such as Ascend NPUs, whose programming model is barely represented in pretraining data. Recent NPU agents address this gap largely through experience: they accumulate memory from their own generation and optimization trials and reuse it on later tasks. We argue that experience alone is not enough for kernel generation. An agent can only remember what it has tried, and it rarely tries a mechanism it does not know. As a result, such memory offers little help when a new operator calls for a different mechanism, and even familiar operators raise unseen situations as optimization proceeds. We present ASCENDEVOLVE, which grounds every step of an evolutionary kernel search in a large external knowledge base built from platform documentation, toolkit headers, and operator repositories. Because this corpus far exceeds any context window, ASCENDEVOLVE indexes it and routes each agent to a small, task-specific slice of evidence instead of the corpus itself. The retrieved knowledge seeds diverse implementation strategies, guides repair when a candidate fails, and constrains how strong candidates are recombined. On 53 Ascend C operators from CANN-Bench spanning four difficulty levels, ASCENDEVOLVE achieves an overall score of 71.39, improving over Reflexion and OpenEvolve by 11.6% and 17.2%.