Tomos format cuts matrix storage without slowing mixed-precision math

TOMOS: Eliminating Data Duplication in Mixed-Precision Matrix Computations

Mathematical Software

Summary

Working with numbers in different precisions often means storing the same data twice, which takes up extra space and time. The authors created TOMOS, a new way to store matrix numbers that merges low and high precision versions into one without losing detail. This lets computers do fast, low-precision calculations and still get exact high-precision results when needed. Their experiments show speed improvements and efficient storage on NVIDIA GPUs without needing duplicate data.

What this means in practice

Authors

Vasileios Georgiou, Petros Drineas

Abstract

Mixed-precision algorithms exploit reduced-precision hardware, however computing with the same matrix at distinct precisions typically requires duplicate storage, repeated conversions, or operation-emulation. We introduce TOMOS, a composite-precision matrix format that provides implicit 32-bit and 16-bit access from a single representation. Each FP32 entry is split into a 16-bit primary word for low-precision computation and a complementary word that enables exact FP32 reconstruction. The encoding scheme improves numerical quality of the low-precision view through subnormal representations, and allows exact recovery of high-precision inputs. TOMOS uses FP32-sized storage, with data movement scaling with compute precision. Optimized GEMM and GEMV kernels achieve throughput comparable to vendor libraries on NVIDIA A100 GPUs, while a mixed-precision least-squares solver achieves up to 1.6x speedup over FP32 without redundant matrix allocations.