Exploiting Semantic and Pixel Representations for Ultra-Low Bitrate Image Compression
2026-06-01 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors designed a new image compression method called SPRDiff that works well even when images are compressed to extremely small file sizes. Unlike older methods that focus mostly on making images look realistic but lose exact details, SPRDiff keeps both the overall meaning and the exact pixel details of the original image. They do this by combining three types of encoded information and using a special module that helps guide the image reconstruction process precisely. Their tests show their method beats existing ones in balancing file size, image quality, and accuracy at very low bitrates.
image compressionrate-distortion-perception trade-offdiffusion modelssemantic representationpixel fidelitylatent compressionentropy modelingVAE encoderbitrate
Authors
Hao Wei, Yanhui Zhou, Chenyang Ge, Saeed Anwar, Ajmal Mian
Abstract
Most existing extreme compression methods fail to achieve an optimal rate-distortion-perception trade-off, as they typically prioritize perceptual fidelity and visual realism over pixel-level accuracy. Consequently, the resulting reconstructions often deviate noticeably from the originals. Ultra-low bitrate image compression is therefore crucial-not only for producing extremely compact representations but also for ensuring that reconstructed images remain semantically coherent and faithful to the source at the pixel level. To this end, we propose SPRDiff, a diffusion-based compression method that fully leverages both semantic and pixel representations, thereby enhancing reconstruction fidelity under ultra-low bitrate constraints. Specifically, we develop a triple-encoder architecture that utilizes high-fidelity features from the pretrained distortion-oriented and semantic-oriented encoders to compensate for the limited representations extracted by the frozen VAE encoder, thereby improving latent compression and entropy modeling. To further enhance the reconstruction fidelity of diffusion models, we introduce a distortion-aware reconstruction module with dual feature extraction. This module not only generates a coarse reconstruction that preserves the main structures, but also provides practical and accurate semantic- and pixel-level conditional signals to guide the diffusion model. Extensive experiments on benchmark datasets demonstrate that our method outperforms state-of-the-art approaches in the rate-distortion-perception tradeoff at extremely low bitrates (below 0.03 bpp), effectively preserving both perceptual quality and pixel-wise fidelity in the reconstructed images. We will release the source code and trained models at https://github.com/cshw2021/SPRDiff.