Controllable sampling method boosts variety in AI generated images
Controllable Diverse Sampling through Pullback Directions
Artificial IntelligenceComputer Vision and Pattern Recognition
Summary
AI models that make images from text prompts usually produce pictures that look very similar even when starting from different random settings, which limits creativity. The paper introduces a new way to steer the image creation process by finding directions where small changes cause big differences in the output. This method does not require extra training and helps create diverse images while still matching the text prompt closely. Tests on image generation and filling in missing parts confirm the method increases variety without losing quality.
What this means in practice
- •For digital artists: Generate multiple distinctive images from the same text prompt to inspire creative workflows without lowering image quality.
- •For graphic design teams: Produce varied inpainting outcomes that maintain visual coherence with the original image for retouching and editing tasks.
Authors
Sylvain Dehayem Kenfouo, Aaron Zweig, Mingxuan Zhang, Elham Azizi
Abstract
Conditional diffusion models can generate high-quality images that align closely with a given prompt. However, samples generated from different random seeds may remain visually similar, limiting creative exploration. Existing approaches can increase variation, but often require additional optimization or sampling overhead, while stronger perturbations may introduce distortions that degrade image quality or prompt alignment. We propose a training-free method that leverages the local pullback geometry induced by the mapping from the conditioning-embedding space to the image space to identify directions of maximal output sensitivity. By steering different sampling trajectories along these directions, we encourage distinct generations while maintaining consistency with the conditioning signal. Experiments on MS-COCO and BrushNet show the efficacy of the method in improving diversity on prompt-conditioned image generation and inpainting.