Generative image compression preserves detail at ultra low bitrates
Rethinking Generative Image Compression at Extremely Low Bitrates
Computer Vision and Pattern Recognition
Summary
Compressing images using AI often leads to bad or unrecognizable results when the file size gets extremely small. The authors found that existing methods struggle to keep important content as the file size shrinks because the goals of reconstruction and semantic understanding conflict. They designed a new method called RAE-CoD that compresses images into very few bits while keeping the images recognizable and naturally structured. This approach prevents sudden quality crashes and makes image compression more reliable even at extremely low bitrates.
What this means in practice
- •For mobile app developers: Create image-sharing apps that compress photos to tiny sizes without losing recognizable content, saving bandwidth and storage on mobile devices.
- •For video streaming engineers: Improve video frame compression to maintain visual coherence at very low bitrates, enabling better streaming over limited networks.
- •For social media platform engineers: Offer tighter image compression while preserving content recognizability, reducing server storage costs without degrading user experience.$Commercial implications: This paper enables commercial deployment of ultra-low bitrate image compression products that retain image quality, appealing to platforms seeking cost-effective content delivery.
Authors
Tianyu Zhang, Zhaoyang Jia, Houqiang Li, Dong Liu
Abstract
Generative image compression produces visually plausible reconstructions at low bitrates, yet their behavior as the rate approaches zero remains largely unexplored. When pushed below normal operating rates, representative codecs undergo semantic collapse: rather than gracefully losing source-specific detail, they produce malformed or unrecognizable content. Our analysis identifies two factors. As the bitrate decreases, reconstruction losses increasingly conflict with semantic objectives on gradients and visual results, while pixel-space and reconstruction-oriented VAE diffusion models become less efficient on semantic preservation. Guided by these findings, we introduce RAE-CoD, a compression-oriented diffusion (CoD) built in a representation autoencoder (RAE) space with direct alignment between compressed and source representations, preserving recognizable, naturally structured content for a $256\times256$ image with as few as 16 bits. We evaluate this framework using five vision foundation models (VFM) and a blinded vision-language model protocol. On MSCOCO-30K, RAE-CoD stands out from all evaluation. At 0.001-0.008 bpp, it reduces relative VFM feature MSE and Fréchet Distance ratio by at least 25.7% and 69.1% over the best competitors. Meanwhile, semantic recognizability and quality of the reconstructions remain nearly constant while source consistency falls smoothly, replacing abrupt semantic collapse with a graceful transition toward unconditional generation. Code will be released at https://github.com/LuizScarlet/RAE-CoD.