Text-to-image models generate more varied images without losing quality
TokenRepel: Generating Diverse Image Sets Without Compromising Quality
Computer Vision and Pattern Recognition
Summary
Sometimes, when computers make pictures from words, they create many images that all look very similar. The authors found a way to help these models produce a wider variety of images for the same text prompt while keeping the pictures looking good. They do this by making the computer representations inside the model push away from each other so the images differ more. They also make sure that the changes don’t make the pictures worse by keeping the computer’s predictions in a high-quality zone. This method works well with different types of text-to-image models.
What this means in practice
- •For graphic designers: Create more varied sets of images from the same text prompt to inspire creativity without extra manual edits.
- •For video game artists: Generate diverse visual assets automatically from text to populate game environments with less repetition.
- •For advertising content teams: Produce multiple distinct marketing visuals from one description to better target different audiences quickly.$Commercial implications: Enables creating varied advertising campaigns efficiently using text-to-image tools, improving customization and speed.
Authors
Qingtao Yu, Zhen Wang, Ning Lai, Boyang Hu, Hongdong Li, Dylan Campbell
Abstract
We present an inference-time approach that enhances the visual diversity of image sets generated by text-to-image models, without compromising the average image quality. Our method leverages the intrinsic geometry of hidden representations in pretrained models to diversify images generated from the same prompt. First, we introduce a token repulsion mechanism that increases the angle between feature tokens across samples. It takes a negative Riemannian gradient step on a von Mises-Fisher mixture distribution of tokens on the unit hypersphere. Second, to preserve the quality of each output image, we propose a token density preservation mechanism that projects the repulsion-induced prediction change onto a half-space aligned with the model's score direction. This encourages the updates to remain within the high-density regions learned by the generative model. We further extend this mechanism to the output of each intermediate layer, enabling finer-grained iterative guidance. We evaluate our method on three types of pretrained text-to-image models. Experiments show substantial gains in visual diversity under identical prompts, while maintaining a comparable level of image fidelity.