Gpu kernel optimization improves by learning adaptive profiling strategies
Less is more: learning what to profile for GPU kernel optimization
Machine LearningArtificial Intelligence
Summary
Optimizing GPU programs often needs information about how they run on the hardware. The authors found that collecting more data about the hardware doesn't always help improve optimization and that the most useful data depends on each program. They created a system called AutoProfiler that learns which hardware details to gather by trying and testing different options automatically. This makes optimization faster and better by focusing on the most helpful information for each program.
What this means in practice
- •For gpu software developers: Automatically customize hardware profiling to improve GPU kernel optimization under limited profiling budgets.
- •For high performance computing teams: Optimize diverse GPU workloads by adapting profiling strategies to extract the most relevant hardware data for each kernel.
Authors
Yihan Wang, Peiyu Liu, Runyu Chen, Wei Xu
Abstract
Large language models have recently shown strong potential for automatically optimizing GPU kernels, yet their effectiveness depends critically on the hardware information available to the kernel optimization model. Existing systems typically rely on predefined profiling strategies, leaving what to profile largely outside the optimization process. We show that effective profiling is not simply about collecting more information: increasing the number of profiling metrics does not monotonically improve kernel optimization, while the most useful profiling information varies across kernels. Our theoretical analysis further shows that, under such heterogeneous profiling utility, kernel-conditioned selection can outperform any single fixed profiling configuration. Motivated by this observation, we propose AutoProfiler, a feedback-driven framework for learning a deterministic profiling tree from optimization feedback. The tree is iteratively refined through REWRITE and SPLIT operations, with an LLM generating candidate modifications and empirical evaluation determining their acceptance. Once constructed, the tree quickly gathers the required profiling information for any kernel, allowing the LLM rewriter to be invoked only once. Experiments on KernelBench show that our approach improves kernel optimization over fixed profiling configurations under constrained profiling budgets. These results suggest that hardware profiling should evolve from static metric collection toward profiling reasoning, making profiling itself a new dimension for improving kernel optimization. The codes are available at https://github.com/Satissss/AutoProfiler.