Papers for

gpu compiler developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Mamba state-space model kernels fit well on programmable CGLA chips

Mamba-Family State-Space Model Kernels on a Programmable CGLA

Abstract: Edge and embedded inference is constrained by power and data movement. Mamba-family state-space models replace attention with sequence-linear recurrence, but their inference path combines dense projections, short-reduction SSD kernels, and recurrent-state updates. This paper maps these kernel groups onto IMAX, a programmable CPU-Grounded Linear Array (CGLA), and measures them from kernel execution to token-level integration. Projection kernels match the long-reduction IMAX pipeline, whereas SSD Step-1 is limited by short reductions and kernel-boundary overheads. Mamba-130M token-level integration identifies projection GEMV as the decode bottleneck. These results show that programmable CGLAs fit long-reduction projection kernels, while SSD and decode-time projection support require boundary reduction and persistent-weight execution.

Wed 23 SeptHardware Architecture
The gist
Running AI models on small devices like phones or sensors is hard because they use a lot of power and data movement. The authors looked at a special type of AI model called Mamba that uses a simpler way to process sequences instead of the usual attention method. They tested how parts of this model run on a new kind of chip that can be programmed to do many small operations in parallel. Their findings show that some parts of the model fit very well on this chip design, while others have performance limits due to the chip’s architecture. This means certain AI model parts can run faster and more efficiently on these chips, but others need special handling.
Open → 2609.27437v1

GPU kernel reductions get predictable bitwise behavior with new techniques

Taming Bitwise Behavior in GPU Kernels with Tensor Core: Black-Box Reconstruction, Compiler Enforcement, and Static Verification

Abstract: Determinism and numerical reproducibility are increasingly required of GPU kernels in machine learning systems, yet deterministic implementations of the same kernel can still differ bit for bit. Floating-point reduction order is the primary cause, alongside partial-sum precision, fused multiply-add operations, and rounding placement. These choices may be hand-coded, selected by a block-level language such as Triton, or hidden inside a closed-source library such as cuBLAS or rocBLAS. A tile shape chosen for performance therefore also determines the arithmetic, potentially breaking batch invariance. Preserving a fixed order can cost up to 20 percent, while an autotuner cannot identify which configurations are bitwise equivalent. We characterize the factors determining the bitwise behavior of reductions and general matrix multiplication (GEMM). First, we introduce a descriptor of GEMM reduction order, including the partitioning of K in split-K GEMM. Using it, we perform the first black-box reconstruction of a closed-source library's arithmetic for bit-level correctness. Our family of Triton GEMMs matches NVIDIA cuBLAS in all tested cases on Blackwell and Hopper. For realistic LLM shapes with fused epilogues, it matches or exceeds torch.compile performance. Second, we enforce balanced-tree reduction during Triton lowering and introduce a data-layout optimization that brings 19 of 27 kernels on GB300 and H100 within 10 percent of free-order performance. Third, we develop sound static checkers for bitwise equivalence between compiled GPU kernels, including the first checker spanning NVIDIA PTX and AMD GCN. Integrated into Triton's autotuner, the checker restricts search to a single bit-equivalence class.

Thu 10 SeptDistributed, Parallel, and Cluster ComputingPerformanceProgramming Languages
The gist
When running math on GPUs, small differences in the order and way numbers are combined can cause tiny but important differences in results. The authors studied what causes these differences, especially in popular GPU math routines. They created a way to understand and replicate the exact behavior of closed-source GPU libraries and made improvements to an open-source compiler to control these differences. They also built tools to check if two GPU programs will produce exactly the same bit-level results. This helps make GPU calculations more consistent and reliable.
Open → 2609.11356v1