Joint smoothing and scaling improve NVFP4 quantization accuracy for AI models
OSFP4: Joint Optimization of Diagonal Smoothing and Block Scales for NVFP4 Quantization
Artificial Intelligence
Summary
Large language models need to shrink how they store and compute numbers to run efficiently. The authors developed a new way to carefully adjust number scales and smoothness to better keep accuracy when using a compact number format called NVFP4. They optimize these adjustments together to reduce errors caused by rounding numbers in the model's calculations. Their tests show this method keeps model accuracy high while running almost as fast as the vendor’s standard method.
What this means in practice
- •For ai model deployers: Use the new quantization method to run large language models efficiently without significantly losing accuracy when storing and computing with smaller numbers.
- •For hardware accelerator engineers: Integrate joint smoothing and scaling optimization into hardware software stacks to improve inference throughput on NVFP4-enabled tensor cores.
Authors
Neriah Ben David, Ori Meir, Or Ordentlich
Abstract
NVFP4 is an attractive datatype for large language model (LLM) inference, offering compact storage and native tensor-core acceleration. However, preserving accuracy using NVFP4 requires careful quantization. In this work we develop a novel quantization scheme called Optimized Smoothing and Scaling for NVFP4 (OSFP4). For each linear projection it uses a diagonal smoothing matrix whose entries are optimized to minimize the squared matrix-product quantization error under NVFP4, taking into account the rounding procedure that is used (either round-to-nearest, or GPTQ-style successive interference cancellation). This requires performing joint optimization on the smoothing entries as well as the block scales, which is facilitated by analyzing a multiplicative-dither FP4 quantizer instead of the fixed deterministic one. Experiments show that OSFP4 achieves the highest average accuracy among the evaluated competitors in the corresponding quantization settings, while retaining approximately 94-97\% of vendor NVFP4 prefill throughput on the measured workloads. Our code is available in https://github.com/neriahbd/OSFP4