Text-to-image method balances quality and variety of generated images
Traversing the Satisfaction-Diversity Frontier in Text-to-Image Diffusion
Artificial Intelligence
Summary
Generating multiple images from a single text prompt often requires a trade-off between how well each image matches what the user wants (reward) and how different the images are from each other (diversity). The paper presents a new approach that requires every generated image to meet a minimum quality level while the whole set remains diverse. This method, called SatisDive, can adjust this balance and helps produce batches of images that are both good and diverse without needing extra training. The authors show that SatisDive consistently outperforms a previous approach in balancing quality and diversity.
What this means in practice
- •For digital artists: Generate batches of images from prompts that simultaneously meet user-defined quality standards and maintain visual diversity to aid creative workflows.
- •For advertising teams: Produce diverse, high-quality image options from single text prompts for marketing materials, improving the selection process in campaigns.
Authors
Kevin Zhai, Siva Rajesh Kasa, Soumya Roy, Sumit Negi, Mubarak Shah
Abstract
Text-to-image generation enables users to explore several images generated from the same prompt. For these generated images to be useful, each one must reflect the user's preferences, measured by a learned reward, and differ visually from the others to maintain diversity. Existing methods are limited: they either address reward and diversity separately or combine them in one aggregate score, enabling high diversity to offset low rewards. In this paper, we address these limitations by formulating generation as satisficing: every image (candidate) must satisfy a reward floor and the batch of images must satisfy a diversity cutoff. The reward floor controls the balance between worst-candidate reward and batch diversity; we show that varying this floor defines a Pareto frontier. To traverse this frontier, we introduce SatisDive, a training-free inference-time method. SatisDive uses a batch-relative reward cutoff to distinguish lower- from higher-reward candidates, emphasizing reward improvement for candidates below the cutoff and diversity among candidates above it. On Pick-a-Pic, at matched DreamSim, SatisDive improves worst-candidate reward over FK steering by up to 0.43 with FLUX.1-dev as the base model and HPSv3 as the reward, and by up to 0.70 with SANA-1.6B as the base model and ImageReward as the reward. More broadly, across their overlapping DreamSim ranges, SatisDive's satisfaction-diversity curve Pareto-dominates FK steering's curve in each setting.