Pixel-space diffusion models improve image generation with feature specialization
Persistence Forcing: Exploiting Feature Specialization in Pixel-Space Diffusion
Computer Vision and Pattern Recognition
Summary
Generating clear images from complex data is tricky because images have broad shapes and tiny details. The paper shows a way to treat different parts of an image differently during the creation process. Some features focus on the big picture while others gradually add fine details. This method helps the model make better, more coherent images by letting stable global features guide detailed features as the image forms.
What this means in practice
- •For graphic designers: Generate high-quality images with clearer global and fine details for creative projects using improved diffusion models.
- •For game developers: Produce realistic textures and images with controlled detail refinement for immersive game environments.
Tested on one dataset.
Authors
Chong Wang, Zixuan Fu, Shiqi Huang, Siyuan Yang, Hao Cheng, Bihan Wen
Abstract
Pixel-space diffusion Transformers (DiTs) directly operate on high-dimensional visual data, yet their hidden representations typically undergo uniform refinement across depth. Natural images, however, are inherently organized at different levels of granularity. Global structure can often be represented compactly, whereas local textures and fine details require richer representations. Motivated by this, we introduce heterogeneous refinement in pixel-space DiTs, assigning different feature groups distinct refinement budgets across depth. Consequently, an ordered feature specialization emerges: sparsely refined features predominantly encode global visual structure, whereas more frequently refined features increasingly specialize toward localized, high-frequency details. We refer to these two groups as persistent and active features, respectively. Building on this emergent specialization, we introduce Persistence Forcing (PerF), which explicitly exploits this persistent--active feature organization for pixel-space image generation. This enables persistent features to continuously condition actively refined features, allowing stable global information to guide the ongoing refinement of finer visual details. During generative sampling, this interaction further induces a meaningful guidance direction that promotes coherent global structure and naturally complements classifier-free guidance. On ImageNet $256\times256$, PerF-L achieves FID of $1.91$, approaching $1.86$ of JiT-H with only half the parameters, while PerF-H further achieves FID of $1.63$ and $1.76$ on ImageNet $256\times256$ and $512\times512$, respectively.