Activation-Weighted Seeded Residual Coding for Low-Bit LLM Weight Repair
2026-08-24 • Machine Learning
Machine LearningComputation and Language
AI summaryⓘ
The authors present Activation-Weighted Seeded Residual Coding (AWSRC), a method to fix errors caused by compressing language model weights into very low-bit formats. AWSRC works by encoding the leftover differences between the original and compressed weights using special seed-generated bases, guided by activation data to focus on important errors. Their approach requires only a small extra storage and improves various quality measures like perplexity and accuracy, getting close to the full-precision model's performance. They tested AWSRC on a 3-billion parameter language model and found it outperforms other compression techniques.
weight quantizationlanguage modelsresidual codingactivation weightingperplexityseed-generated basescompression codecINT4BF16model accuracy
Authors
Zehao Liu, Chuangchuang Fang, Yang Ren
Abstract
Low-bit weight quantization saves storage but leaves errors that degrade language-model quality. We introduce Activation-Weighted Seeded Residual Coding (AWSRC), a compact repair codec for an existing quantization backbone. Given a reconstructed weight $W_0$, AWSRC encodes the residual $W-W_0$ using deterministic seed-generated bases. The sidecar stores seed selectors, low-bit coefficients, and scales rather than an explicit codebook. Activation statistics prioritize errors that affect layer outputs. On Qwen2.5-3B-Instruct, adding 0.162 scope-bits/weight to an INT4 RTN backbone closes 88.2%, 78.9%, and 71.3% of the matched PPL, KL, and accuracy gaps to BF16. Repairing a matched strong low-bit backbone also improves all measured quality metrics. With a matched 49.25 MB sidecar, about 0.8% of the BF16 model-weight payload, AWSRC gives the best perplexity and mean task accuracy among sparse, low-rank, and vector-quantized codecs.