Tomos format cuts matrix storage without slowing mixed-precision math
TOMOS: Eliminating Data Duplication in Mixed-Precision Matrix Computations
Mathematical Software
Summary
Working with numbers in different precisions often means storing the same data twice, which takes up extra space and time. The authors created TOMOS, a new way to store matrix numbers that merges low and high precision versions into one without losing detail. This lets computers do fast, low-precision calculations and still get exact high-precision results when needed. Their experiments show speed improvements and efficient storage on NVIDIA GPUs without needing duplicate data.
What this means in practice
- •For high-performance computing teams: Run mixed-precision matrix multiplications faster with fewer data copies on GPUs using TOMOS format.
- •For machine learning engineers: Build faster mixed-precision least-squares solvers without extra memory overhead to accelerate model training.
Authors
Vasileios Georgiou, Petros Drineas
Abstract
Mixed-precision algorithms exploit reduced-precision hardware, however computing with the same matrix at distinct precisions typically requires duplicate storage, repeated conversions, or operation-emulation. We introduce TOMOS, a composite-precision matrix format that provides implicit 32-bit and 16-bit access from a single representation. Each FP32 entry is split into a 16-bit primary word for low-precision computation and a complementary word that enables exact FP32 reconstruction. The encoding scheme improves numerical quality of the low-precision view through subnormal representations, and allows exact recovery of high-precision inputs. TOMOS uses FP32-sized storage, with data movement scaling with compute precision. Optimized GEMM and GEMV kernels achieve throughput comparable to vendor libraries on NVIDIA A100 GPUs, while a mixed-precision least-squares solver achieves up to 1.6x speedup over FP32 without redundant matrix allocations.