A GPU-Accelerated Blocked Adaptive Randomized Range Finder Based on an Implicit Householder QR Decomposition
Mathematical Software
Summary
The authors propose a new way to simplify big math problems used in training deep neural networks by breaking them into smaller, easier parts using a technique called a randomized range finder. Their method improves stability and speed by using a mix of CPU and GPU computing and avoids some common pitfalls in previous methods. Tests showed it works well and is much faster, especially on powerful GPUs like the NVIDIA GH200. This approach can help make complex scientific and engineering computations more efficient.
Authors
Carolin Penke, Andreas Herten
Abstract
Low-rank methods can reduce the memory and computational requirements of deep neural network training in approaches such as GaLore. Randomized range finders offer an attractive alternative to singular value decompositions, particularly when the required rank is determined adaptively from a prescribed approximation tolerance. We introduce a blocked adaptive randomized range finder based on an implicit Householder QR decomposition and an optimized hybrid CPU--GPU implementation. The proposed method avoids explicit reorthogonalization. Numerical experiments show that it preserves orthogonality and approximation accuracy in regimes where block Gram--Schmidt without reorthogonalization becomes unstable. The blocked formulation exposes matrix--matrix operations and enables overlap of CPU panel factorization with GPU updates. On an NVIDIA GH200, the overlapped implementation reduces the runtime for the largest tested matrix from 9.91 seconds on the CPU to 0.407 seconds. The method provides a stable and efficient building block for low-rank approximation on heterogeneous systems with applications in computational science and engineering.