Mixed precision quantization improves large language model accuracy
Beyond Scalar Sensitivity: Activation-Aware Mixed-Precision LLM Quantization with Cross-Layer Refinement
Machine Learning
Summary
Large language models (LLMs) need to shrink their data size by reducing precision in how their numbers are stored. Previous methods used simple guesses for how sensitive different parts of the model are to this reduction, but these guesses can be very off, leading to worse results. The authors showed the math behind this problem and created a new method that looks at how the model’s parts actually activate together and checks how changes affect overall performance. Their approach lets models keep much more accuracy, especially when using very low precision.
What this means in practice
- •For ml engineers: Implement efficient mixed-precision quantization to reduce LLM size while maintaining accuracy, especially useful at ultra-low bit widths.
- •For ai hardware designers: Design hardware that better supports activation-aware mixed-precision schemes to optimize LLM performance and resource use.
Authors
Akihiro Yoshida, Yuma Ichikawa
Abstract
Mixed-precision weight quantization is commonly formulated as a Multiple-Choice Knapsack Problem (MCKP), yet existing solvers rely on scalar sensitivity proxies that collapse each weight matrix's Hessian into a single number and treat every module independently. We prove that even the optimal scalar proxy incurs multiplicative distortion up to $\sqrt{κ(\mathbf{A})κ(\mathbf{B})}$ relative to the full activation-aware quadratic, where $κ(\mathbf{A})$ and $κ(\mathbf{B})$ denote the condition numbers of the input- and output-side Hessian factors. This bound varies from $10^1$ to $10^{13}$ for typical LLM modules, making inter-module sensitivity ranking unreliable. To address these limitations, we propose Cross-layer Activation-aware Sensitivity Allocation (CASA), a two-phase method. In Stage 1, the scalar proxy is replaced by an activation-aware metric derived from the Kronecker-factored Hessian, reducing the MCKP to a form whose continuous relaxation admits a closed-form solution. In Stage 2, a cross-layer-aware local search evaluates bit-width updates using the end-to-end model loss. Experiments on multiple LLMs across different bit budgets show that CASA achieves lower perplexity than the latest scalar-proxy baselines, especially at ultra-low bit-widths ($<3$ bits per weight). Moreover, the performance gain in zero-shot accuracy tracks the per-model average condition-number over modules, confirming the distortion bound as a practical indicator of scalar-proxy failure.