EntroPack compresses neural network weights efficiently with flexible bitrates
EntroPack: Fast and Accurate Entropy-Coded Weight Compression at Arbitrary Bitrates
Machine LearningArtificial Intelligence
Summary
Large neural networks often use a lot of memory because their weights are stored in fixed sizes. The authors developed EntroPack, a method that compresses these weights more precisely by adjusting how many bits are used, not just using fixed sized blocks. This method lets computers restore the weights quickly during use, without extra training or adjustments. Tests show it reduces storage needs and errors better than older methods.
What this means in practice
- •For machine learning engineers: Reduce memory for model weights in compute-heavy tasks like diffusion denoising or Transformer prefill with minimal performance loss.
- •For graphics software developers: Compress and decompress image generator weights more accurately to improve storage efficiency while maintaining output quality.
Authors
Hong Zhang, Zhongjie Duan, Yingda Chen
Abstract
Weight compression helps large neural networks fit deployment memory budgets, but common fixed-width formats offer only coarse storage choices. Entropy coding supports finer rates, yet the achieved size depends on the quantized weight distribution and coding overhead. Exploiting this flexibility requires accurate rate selection and efficient weight reconstruction for inference. We present EntroPack, an entropy-coded weight compressor that supports arbitrary target bitrates without activation calibration or fine-tuning. It combines row-normalized $E_8$ lattice quantization with a conditional probability model of lattice coordinates. Sampled storage estimates select the quantization resolution without repeated full-stream encoding. The final coordinates are entropy-coded in independently decodable tiles, enabling fast, fused symbol decoding and numerical weight reconstruction on the GPU. EntroPack supports floating-point and integer weight containers, such as BF16, FP16, FP8, and INT8, with storage bitrate controlled independently of numerical precision. Online decoding adds latency that grows with weight count, making the method well suited to compute-intensive workloads such as diffusion denoising and Transformer prefill. Experiments demonstrate fast encoding and modest inference overhead in these settings. When compressing the linear-layer weights of the image generator Z-Image-Turbo, EntroPack achieves substantially lower weight and denoiser output errors than fixed-width formats at comparable storage rates, with modest denoising-step overhead. Targeting 4 bits per parameter, it achieves lower weight and denoiser output errors than NF4, including about 24% lower relative $L_2$ weight error, with less storage. Source code is available at https://github.com/modelscope/entropack.