Bitstream native restorer improves corrupted image recovery accuracy
BiNar: Bitstream Native Restorer for Bitstream-corrupted Image Restoration
Computer Vision and Pattern Recognition
Summary
Sometimes images get damaged because their compressed data files have errors that normal decoders can't fix, leading to blurry or broken pictures. The authors created BiNar, a tool that works directly with the corrupted data to understand its structure before trying to fix the image. It combines this deep data insight with visual clues from the partly corrupted images to better locate and repair the damaged parts. This approach allows their method to restore images more accurately than existing techniques, even when the data is badly corrupted.
What this means in practice
- •For video streaming providers: Improve image quality recovery in video streams suffering from bitstream errors to enhance viewer experience without retransmission.$Commercial implications: Enables streaming platforms to sell higher quality, error-resilient video services by reducing visible corruption from transmission errors.
- •For digital forensic teams: Recover partially corrupted compressed images used as evidence by extracting and repairing bitstream-level errors for clearer analysis.
Authors
Fangcheng Li, Zhen Yu, Kejun Wu, Tianyi Liu
Abstract
Bitstream-corrupted image restoration (BCIR) aims to reconstruct faithful images from corrupted compressed bitstreams. However, corruption occurs in bitstream-domain may lead to unpredictable error propagation and distortion amplification in pixel-domain, and even result in complete undecodability by standard decoders, posing significant challenge to existing image restoration paradigms. To address this challenge, we propose Bitstream Native Restorer (BiNar), a generative restoration framework that leverages fine-grained bitstream-native semantics to guide bitstream-corrupted image restoration. In BiNar, Bitstream Semantic Modeling (BSeM) extracts robust informative semantics and models sequential bytes directly from corrupted bitstream, enabling to generate fine-grained bitstream descriptions without relying on pixel decoding. Based on the descriptions,the Corruption-adaptive Multimodal Semantic Aggregation (CMSA) further extracts adaptive visual modal cues in decoded corrupted images and aggregates to bitstream descriptions. The multimodal aggregated fine-grained descriptions can serve as coherent restoration guidance. Finally, Blind Corruption Localization (BCoL) exploits the segmentation prior of visual foundation models to identify and locate error regions that requires recovery in corrupted images. The located error regions and the fine-grained descriptions jointly serve as visual prompt and language prompt of diffusion models image restoration, respectively. Extensive experiments under diverse bitstream corruption conditions show that BiNar outperforms all State-of-the-arts image restoration models regardless of perceptual quality or objective quality. Ablation studies demonstrate the effectiveness of fine-grained bitstream-native semantics in BCIR task.