Diffusion model improves real-world image super-resolution with dual inputs
DNF-SR: Dual-Input and Negative-Aware Feature Fine-Tuning for Real-World Image Super-Resolution
Computer Vision and Pattern Recognition
Summary
Making low-quality images clearer and more detailed is hard, especially in one step. The authors found that feeding just a noisy low-quality image into a special AI called a diffusion model can mess up the details. So, they combined the original image with a noisy version and used a new technique to help the model learn which results are better or worse. This method helps produce sharper and more consistent high-resolution images in one step.
What this means in practice
- •For mobile app developers: Improve mobile photo enhancement apps by integrating a one-step super-resolution method that preserves more detail and clarity in real-world low-resolution images.
- •For video streaming engineers: Enhance video quality by upscaling low-resolution frames efficiently with better visual consistency using dual-input diffusion-based models.
Authors
Shuhao Han, Wenjie Liao, Hayden Vance, Hang Dong, Rui Zhang, Chun-Le Guo, Chongyi Li
Abstract
Benefiting from the powerful generative priors of diffusion models, diffusion-based real-world image super-resolution (Real-ISR) methods have demonstrated impressive performance.To achieve efficient Real-ISR, several recent works have designed one-step diffusion-based models.Howerver, unmediatedly feeding LR into a diffusion model creates a distributional gap with the model's original input.A straightforward approach to reduce the distribution gap is to introduce noise to the LR latents. However, directly adding noise inevitably corrupts the content of the LR images.In this study, we propose DNF-SR, a Dual-input and Negative-aware Feature fine-tuning method for Real-ISR.Specifically, we use a dual-input strategy that concatenates the original LR image with the noisy LR input and feeds them into a diffusion-based image editing model, ensuring both high-fidelity one-step super-resolution and improved perceptual and content consistency.Additionally, the noise present in the noisy LR input introduces randomness and diversity into the outputs. We exploit this property and propose a post-training optimization method, Negative-aware Feature Fine-Tuning (NF2T), which guides the model toward producing higher-quality results.NF^2T classifies multiple outputs into positive and negative subsets and then defines implicit policy improvement directions in both the image and feature spaces, thereby further enhancing the stability of the optimization.Extensive experiments show that DNF-SR outperforms other methods.Code will be released.