LLM driven tool improves edge device kernel generation and optimization
EdgeKernelBench: Unified LLM-Driven Kernel Generation for Edge Backends
Machine Learning
Summary
Generating efficient computer instructions (kernels) for edge devices is hard because these devices are different and have limited resources. The authors created EdgeKernelBench, a way to test and compare how well large language models (LLMs) can generate these kernels across many types of edge devices. They found that simply generating correct kernels doesn’t guarantee good performance, so they built LEKA, a tool that optimizes kernels step-by-step and performs better. Together, these tools help developers evaluate and improve how AI creates code for edge devices.
What this means in practice
- •For edge device developers: Evaluate and optimize AI-generated kernels for different edge hardware to improve application performance.
- •For compiler engineers: Use a unified IR framework to connect kernel generation to multiple edge backends for streamlined backend support.
- •For mobile game developers: Improve the performance of AI-driven graphics and computation on diverse mobile devices using layered kernel optimization.$Commercial implications: Enables selling performance-enhanced mobile games optimized for many devices using improved kernel generation.
Authors
Yuhua Zheng, Xueyi Chen, Xinze Zou, Xin Li, Haoji Hu, Huan Wang
Abstract
Recent works show that LLMs can generate kernels with strong performance on server GPUs. Edge deployment presents distinct challenges because the hardware is heterogeneous, the software stacks are fragmented, and the resource budgets are tight. LLM kernel generation across diverse edge backends has yet to be studied systematically. We introduce EdgeKernelBench, which combines a unified IR deployment framework with a hierarchical benchmark for edge workloads. The unified IR captures reusable computation, scheduling, and memory decisions and connects to each target backend through a compiler interface. The benchmark spans progressively larger program scopes, from individual operators to compact model graphs, across representative edge application domains. We instantiate the design with TVM as a reference compiler and validate it across heterogeneous edge backends. With a single generation attempt, the best model achieves 64.5% Correct@1 but only 25.5% Fast@1.0. This gap shows that functional correctness does not consistently translate into performance gains. We therefore introduce the Layered Edge Kernel Agent (LEKA), which organizes optimization through a layered IR. On the agent evaluation subset, LEKA achieves 43.5% Fast@1.0, compared with 13.0% for direct generation and 26.1% for Claude Code. LEKA and Claude Code use the same base model and optimization budget. Together, EdgeKernelBench and LEKA support unified evaluation and layered optimization of kernels generated by LLMs across edge backends.