H-Scale: Hessian-Guided Scale Refinement for NVFP4 Sub-Byte LLM Inference
Computation and Language
Summary
The authors introduce H-Scale, a new method to improve the way NVIDIA's Blackwell GPUs handle a special number format called NVFP4 when running large language models. NVFP4 splits weights into small groups with individual scales, but choosing the best scales is tricky and hasn't been studied much. Instead of just trying to match the original weights, the authors’ approach uses information from the model’s activations to pick scales that better preserve the model's output. This method is easy to add to existing systems, needs little extra setup, and doesn’t slow down the model when it runs. Their tests show H-Scale generally makes the model’s accuracy closer to a higher-quality standard.
Authors
Hao Yu, Zheng Li, Dayiheng Liu, Jianwei Zhang
Abstract
The NVIDIA Blackwell architecture, with native support for the ultra-fine-grained NVFP4 format, opens new opportunities for accelerating large language model (LLM) inference. NVFP4's micro-block design, such as a group size of 16, offers strong representational flexibility for capturing local weight distributions and isolating outliers, but it also introduces a large and highly sensitive space of per-group scaling factors. Existing post-training quantization (PTQ) methods primarily focus on refining quantized weight values, leaving this scale-selection step underexplored. To address this gap, we propose \textbf{H-Scale}, a lightweight post-processing method for NVFP4 per-group scale refinement. Instead of minimizing plain weight reconstruction error, H-Scale selects hardware-valid group scales using a diagonal second-order proxy derived from calibration activations, thereby targeting layer output perturbation more directly. It is designed as a drop-in replacement for RTN-style scale selection in diverse NVFP4 pipelines, requires only modest offline calibration, and introduces strictly zero overhead at inference time. Under a fixed evaluation protocol, experiments on mainstream LLMs show that H-Scale generally improves a broad range of NVFP4 baselines and brings several variants closer to the BF16 reference.