Diffusion models improve text accuracy in image super-resolution
TGSR: Text-Grounded Diffusion for Faithful Image Super-Resolution
Computer Vision and Pattern Recognition
Summary
Making blurry pictures clearer often causes text in the image to look wrong or fake. The authors found that this happens because current methods try to improve text everywhere without checking if it's reliable. They propose a way to only sharpen text that is likely correct and avoid making up fake words. Their method uses a special detector to find and reduce these errors, leading to clearer and more accurate text in enhanced images.
What this means in practice
- •For document digitization teams: Enhance scanned documents by improving text clarity while reducing false or distorted characters in image upscaling.
- •For advertising content creators: Generate high-resolution images with accurate text elements to maintain brand integrity in digital marketing materials.
Authors
Zhiming Hu, Ran Zhang, Angela Ye, Raghav Goyal, Faraz Ali, Tristan Ty Aumentado-Armstrong, Samrudhdhi B. Rangrej, Konstantinos G. Derpanis, Iqbal Mohomed, Alex Levinshtein
Abstract
Diffusion models have achieved remarkable success in image super-resolution (SR) by leveraging strong generative priors to enhance perceptual quality. However, for textual content, these priors often produce text hallucinations: non-existent or distorted text, which alter scene semantics and compromise model reliability. While recent methods focus on enhancing text generation, they do so indiscriminately, leaving hallucinations fundamentally unresolved. We argue the core problem is a lack of selective control. Therefore, we reformulate text SR as a selective generation problem: text is resolved only when reliable generation is expected, while other scene content is fully super-resolved. We introduce TGSR (Text-Grounded Super-Resolution), a test-time framework that progressively identifies and suppresses likely hallucinated text regions during generation. To enable this, we train dedicated detectors on a large-scale dataset curated from diverse diffusion-based SR outputs. TGSR consistently improves text fidelity across multiple backbones, increasing recognition precision by up to 17.68 points without degrading perceptual quality, underscoring the value of selective control in generative SR.