Diffusion transformers produce clearer high resolution images with new attention and guidance
Restoring Semantic Fidelity for Training-Free High-Resolution Image Generation in Diffusion Transformers
Computer Vision and Pattern Recognition
Summary
Generating large, high-resolution images with diffusion transformers often leads to important details and meaning getting lost. The authors propose a new method that keeps the text guidance global while adjusting the image details to match what the model was trained on, restoring the meaningful content. They also introduce a way to guide the image creation process by focusing separately on structure and fine details over time. Their approach improves how well the images keep their shapes and intricate features when made bigger without retraining the model.
What this means in practice
- •For digital artists: Produce clearer and more detailed large images from text prompts without needing costly retraining of models.
- •For creative app developers: Create apps that generate high-resolution, semantically coherent images on the fly by improving diffusion transformer outputs.$Commercial implications: Enables high-quality image generation tools that deliver better output without extra training, attractive for design and media software markets.
Authors
Junjie Li, Zengbin Wang, Gaoxiang Li, Muyi Sun, Yixin Zhang, Man Zhang
Abstract
Diffusion Transformer (DiT)-based text-to-image models have recently achieved remarkable success in image synthesis. However, when directly applied beyond their training resolution, these models often suffer from severe semantic degradation, where meaningful semantic content disappears. Although existing training-free high-resolution generation methods can mitigate this issue, the inherent semantic degradation at high resolutions remains largely unresolved, leading to degraded fine-grained details. Furthermore, widely used structural guidance methods lack modeling of the structure and details in the intermediate trajectory, which degrades the quality of the generated results. In this paper, we propose a training-free framework for high-resolution image generation to address these challenges. First, we propose a text-visual scale-decoupled attention (TVSDA) module, which restores the model's ability to generate semantically meaningful content at high resolutions by preserving the global receptive field of text tokens while aligning the receptive field of image tokens with the training resolution. Second, we propose a structure-detail guidance (SDG) strategy that explicitly models structural information and fine details through scale-wise and temporal-wise guidance, overcoming the limitations of conventional low-resolution guidance. Extensive experiments demonstrate that our method significantly improves both structural coherence and detail fidelity over existing training-free DiT-based high-resolution generation approaches. The code is available at https://github.com/ljjcoder/Restoring-Semantic-Fidelity.