Diffusion transformers enhance real-world image super-resolution efficiently
Fill2SR: Repurposing Inpainting Diffusion Transformers for Real-World Super-Resolution
Computer Vision and Pattern Recognition
Summary
Improving low-quality images to higher resolutions is usually costly in memory and limited by fixed scales. The authors introduce Fill2SR, which cleverly reuses an image-inpainting diffusion transformer to enhance images without extra memory-heavy parts. They also develop a way to learn about real image degradations and apply that knowledge to improve clean images. Their method works well on different image sizes and runs efficiently on common hardware.
What this means in practice
- •For image processing engineers: Improve real-world image quality consistently across various sizes without added memory costs by integrating Fill2SR into restoration pipelines.
- •For content creators: Enhance low-resolution or degraded images to high-quality outputs for visual media projects using a stable, scalable method.
Authors
Xingfu Yi, Xiaoxue Yu
Abstract
Recent real-world image super-resolution (SR) methods often adapt text-to-image (T2I) backbones with ControlNet-style branches or spatial conditioning tokens, which increases memory and computes with resolution and often constrains training to a fixed scale. We propose Fill2SR, which repurposes a masked-inpainting Diffusion Transformer for SR without extra spatial branches. Our Inpainting-Interface Evidence Adapter (IIEA) writes the low-quality (LQ) observation into the native masked-image slot under a full-image mask, turning inpainting into a reverse-degradation conditional rectified flow trained with LoRA-only tuning. We further introduce RCDT, an offline pipeline that distills degradation descriptors from unpaired real images and transfers them onto clean targets using frozen open-source models. Fill2SR supports mixed-resolution training up to QHD and yields stable performance across $512/1024/2048$ outputs. On synthetic benchmarks, our base model with IIEA achieves the best LPIPS on DIV2K and LSDIR; adding RCDT trades a small LPIPS drop for consistently stronger no-reference quality on RealLQ250 and RealPhoto60. Fill2SR remains memory-predictable, running $1536^2$ inference on a single 32GB GPU and extending to multi-megapixel outputs via tiled restoration.