Gpu performance varies for low order finite element computations
Performance Analysis of Low-Order, GPU-accelerated Finite Element Kernels using Kokkos
Performance
Summary
Finite element methods help solve complex physics problems by breaking them into smaller parts, but running these calculations efficiently on different GPUs can be tricky. The authors studied a specific low-order computation used in earth science models and tested how it runs on several new GPUs from NVIDIA, AMD, and Intel. They found that the performance depends a lot on how the calculations fit into the memory system of each GPU. Their work shows ways to tweak the code to run better across different GPUs using a programming tool called Kokkos.
What this means in practice
- •For high performance computing engineers: Optimize finite element simulation codes for better performance across diverse modern GPUs using tuning strategies demonstrated for Kokkos kernels.
- •For earth science model developers: Improve runtime efficiency of geophysical PDE simulations by applying the studied memory and parallelism optimizations on GPU platforms.
Authors
Fabian Böhm, Nils Kohl, Harald Köstler, Ulrich Rüde
Abstract
We study performance portability for low-order, matrix-free finite element kernels, using the example of a vectorial, variable-coefficient PDE operator originating in geophysical models. Written in Kokkos, the kernel is compared on NVIDIA H100, AMD MI250X, AMD MI300A and Intel PVC Max 1550 GPUs. Owing to its low order and to optimizations that reduce the arithmetic, the kernel has a low arithmetic intensity, so that its performance is determined by how the finite element assembly is mapped onto the memory hierarchy. This is a dimension in which the architectures differ even within one vendor family, causing different performance characteristics. We examine how Kokkos' hierarchical parallelism and shared scratch memory, which are used for the shared degrees of freedom of the conforming discretization, behave on each device. Finally, we show how portability gaps can be narrowed with tuning levers such as the size of the thread groups, the balance between occupancy and register use, and the atomic accumulation strategy at the end of the kernel.