Resarc improves ultra-low bitrate image compression with residual correction

ResARC: Residual-Aware AutoRegressive Coding for Ultra-Low Bitrate Image Compression

Computer Vision and Pattern Recognition

Summary

Compressing images into very small file sizes often leads to quality loss because some details are lost when the image data is simplified. The paper presents ResARC, a method that helps fix these losses by predicting and correcting the missing details during decompression. It uses two steps to handle different types of errors—one to guess lost details from the data sent and another to send extra helpful information to fix mistakes. This approach helps create images that look better and match the original details more faithfully at extremely low bitrates.

What this means in practice

  • For mobile app developers: Reduce image data size in mobile apps to improve load times and save bandwidth without severely sacrificing image quality.
  • For video streaming platforms: Compress thumbnail and preview images at ultra-low bitrates while maintaining visual fidelity to enhance user experience on low-bandwidth connections.

Authors

Qin Yan, Ruixiao Dong, Yutao Xie, Li Li, Ying Chen, Kai Li, Daowen Li, Houqiang Li

Abstract

Progressive autoregressive image codecs provide an appealing paradigm for generative compression by quantizing continuous latents into discrete tokens, transmitting coarse-to-fine prefix tokens and generating the remaining suffix tokens at the decoder. However, their reconstruction quality is fundamentally limited by two residuals introduced along this pipeline: the quantization residual, arising from information loss during discrete tokenization, and the generation residual, resulting from imperfect autoregressive generation of the suffix tokens. To address these limitations, we introduce ResARC, a residual-aware autoregressive codec that explicitly compensates for both residuals at the decoder. Specifically, we generate the quantization residual with a diffusion transformer conditioned on the autoregressive decoding context, while requiring no additional side information. In parallel, we compute the generation residual at the encoder and employ a learned Generation Residual Codec to efficiently compress and transmit it for decoder-side correction. The recovered residuals are then integrated with the reconstructed latent representation and decoded through an adapted VAE decoder. Extensive experiments demonstrate that ResARC achieves competitive perceptual similarity while substantially improving distributional fidelity over leading generative codecs across the ultra-low bitrate regime. Code and models will be released soon.