AutoRegressive image generation models made efficient with new quantization
HGA-Quant: Heterogeneity-Aware Quantization for AutoRegressive Visual Generation Models
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
High-quality image generation models usually need lots of computation and memory, making them hard to use on small devices. The authors propose a new method to shrink these models without any extra training, keeping the images looking great even with fewer resources. This method focuses on better handling rare but important features in the model and smartly adjusting precision during image creation. Their approach works well on various models, producing near-original quality images with much smaller memory use than before.
What this means in practice
- •For mobile app developers: Enable high-quality image generation directly on mobile and edge devices using smaller, more efficient ARVG models with maintained image quality.
- •For graphics hardware engineers: Design custom hardware accelerators that use mixed-precision quantization schemes tailored to autoregressive image models for improved efficiency.
Authors
Xinmeng Liu, Xianglong Yan, Kaicheng Yang, Weisheng Dong, Yulun Zhang
Abstract
Recently, AutoRegressive Visual Generation (ARVG) models have emerged as a competitive paradigm for high-quality image synthesis. However, their substantial computational and memory demands make them difficult to deploy on resource-constrained edge devices. Post-Training Quantization (PTQ) provides a promising solution to reduce the computational and memory costs of ARVG models, yet existing methods often suffer from substantial image quality degradation under low-bit quantization. In this paper, we propose a training-free PTQ framework for ARVG, consisting of Tail-Aware Bounded-init Grid Refinement (T-BGR) and Coarse-to-Fine Activation Precision Scheduling (CAPS). T-BGR introduces a tail-aware reconstruction objective for weight quantizer calibration, assigning greater importance to distribution-tail coefficients and preserving fine-grained information without introducing additional parameters or high-precision branches. CAPS performs coarse-to-fine activation precision scheduling by jointly considering the importance and computational cost of different generation stages, enabling efficient mixed-precision allocation under a given bit budget. Extensive experiments on multiple ARVG backbones demonstrate that our framework consistently improves robustness under aggressive low-bit quantization. Notably, on Infinity, our framework achieves near full-precision image generation quality at an effective W4A5.6 activation precision, while existing methods at W4A6 still exhibit noticeable quality degradation or inconsistent performance. Code is available at https://github.com/lxm-lc/HGA-Quant.