SyclKittens boosts Intel GPU AI kernel performance by encoding efficient methods

SyclKittens: A Tile Programming Model for Programmers and Coding Agents on Intel GPUs

Distributed, Parallel, and Cluster Computing

Summary

Writing fast programs for new AI chips is hard because it needs deep knowledge of the hardware. The authors show that when coding agents get simple, low-level interfaces, they produce slower programs. But if these agents use SyclKittens, a special programming model with expert tips built-in, they can create much faster programs on Intel GPUs. This leads to big speedups for AI tasks like running Llama models compared to existing tools. The new model is open source for others to use and improve.

What this means in practice

  • For gpu kernel developers: Create high-performance GPU kernels for AI workloads on Intel GPUs by using a programming model that encodes hardware-specific optimization methods.
  • For ai infrastructure teams: Deploy faster AI model inference on Intel GPUs by integrating kernels optimized with SyclKittens into inference systems.

Authors

Yehong Jiang, Sheng Chen, Fangwen Fu, Yen-Kuang Chen, Xinmin Tian, Stuart H. Sul, Simran Arora

Abstract

New AI accelerators arrive before the kernels that make them fast, because peak kernel performance requires architecture-specific expertise in operand pipelines and data-movement techniques. Coding agents can now write, compile, and tune kernels on their own, so they could greatly accelerate kernel development and optimization. What agents produce depends on the interfaces they are given. These interfaces may expose a machine's raw capabilities or encode the known-good methods for exploiting its hardware features efficiently. We test the performance impact of these two interfaces on Intel GPUs through controlled experiments with three coding models on three kernels. With raw SYCL and execution feedback, Opus 4.8, the strongest of the three models tested, writes a single-GPU GEMM kernel reaching only 57.5% of Intel's tuned oneDNN library. We present SyclKittens, a hardware-aware tile programming model for Intel GPUs that encodes these known-good methods, so its operations place operands in matrix-engine layouts, prefetch through the L1 cache, and communicate over the fabric that links GPU stacks into a node. With SyclKittens under the same agent, task, and feedback budget, the agent reaches 82.1%, showing that encoded methods turn hardware capabilities into performance. Workload-specific policies stay programmable, and engineers and agents jointly refine these schedules in SyclKittens to build a kernel suite that reaches ~96% of oneDNN in geometric mean across GEMM shapes. The suite runs Llama-3.1-8B inference 1.59x faster than torch.compile on one GPU and up to 2.91x faster than a matched multi-GPU decode path on Intel's oneCCL. SyclKittens is open source and available at https://github.com/intel/SyclKittens.