Tensor operator speed varies across GPUs due to instruction costs
Instruction-Aware Transfer of GPU Implementation Crossovers: a Tensor-Product Discontinuous Galerkin Operator on GB200, H100, and A100
PerformanceDistributed, Parallel, and Cluster Computing
Summary
Running a certain mathematical operation called a tensor-product discontinuous Galerkin operator runs differently on three GPU models: GB200, H100, and A100. The fastest way to run it depends on the machine because the cost of individual instructions varies with each GPU’s design. The authors created a method to estimate the true performance by assigning each instruction a cost based on the GPU's characteristics, allowing them to predict which GPU and method will be fastest. Their model closely matches real times without needing to fit parameters to the targeted GPU’s data.
What this means in practice
- •For gpu performance engineers: Predict fastest implementation paths on new GPUs without detailed on-target profiling by computing instruction-dependent costs.
- •For scientific computing software developers: Optimize high-precision numerical kernels for different GPU architectures by understanding instruction cost tradeoffs.
Authors
Seiya Nishizawa, Yuta Kawai, Xuanzhengbo Ren
Abstract
We compare up to six implementations of an FP64 tensor-product discontinuous Galerkin operator at p = 7-1023 on GB200, H100, and A100 with fixed numerics, input, and code. The fastest path is machine dependent: at p = 511, a fused-epilogue GEMM path is 5.0% faster than a fused Tensor Core kernel on GB200 and 6.9% slower on A100. Nominal-peak scaling reproduces the A100 ranking but fails on H100, where the cost of FP64 arithmetic in the profiler-defined FP64 budget depends on the instruction. We therefore price each issued instruction at its architecture-dependent cost in that budget, using a calibration-side counter profile and one derived calibration factor per kernel, with no fit to the target. A retrospective leave-one-GPU-out evaluation, holding out target timings and counters but not the model form, transfers the p >= 511 path times with 1.9% median error at the common MMA shape, and the direction of the crossover at the H100 shape. Restricting the streaming multiprocessors of one A100, with nothing fitted, reproduces the A100-to-GB200 reversal. At roduction-relevant orders the model does not resolve the fastest fused path; the result identifies the crossover mechanism.