Diffusion models learn to generate images matching personal user preferences
Personalize at Test Time: Learning User Preferences for Image Generation
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Generating images that match what different people want is hard because everyone likes different things. The authors created a way for a computer to learn what each person prefers just by looking at the pairs of images people say they like more. They shrink lots of image details into a smaller set of traits, then figure out how each user weighs these traits to pick their favorite. This helps the computer make pictures a user will like without retraining the whole system every time.
What this means in practice
- •For graphic designers: Customize image outputs quickly by tuning user preference weights to reflect client feedback without retraining the entire model.
- •For mobile app developers: Integrate personalized image generation that adapts to user tastes from minimal preference feedback during app use.$Commercial implications: Enables personalized content creation features in consumer apps, improving user engagement and satisfaction directly.
Authors
Jiamu Bai, Jiaming Hu, Yanhong Wu, Zellux Wang
Abstract
Diffusion models can generate high-quality images, yet aligning their outputs with individual user preferences remains challenging. A key bottleneck is accurately modeling diverse user preferences from limited feedback. Existing approaches often rely on labor-intensive manual preference annotations or vision-language models (VLM) to extract preference information from user interaction histories, introducing substantial annotation or computational costs that limit scalability. We propose an approach that learns personalized reward models directly from users' historical image preference pairs. First, we use an autoencoder to compress hundreds of visual attributes into 50 attribute-anchored preference dimensions and train an evaluator to score images along these dimensions. We then represent each user's preferences as a linear combination of the shared dimension scores, estimating the user-specific weights by maximizing the likelihood of their observed pairwise preferences under the Bradley-Terry model. This formulation reduces per-user adaptation to optimizing a low-dimensional weight vector, simplifying optimization and enabling data-efficient personalization from sparse feedback. The learned personalized rewards guide image generation at inference time while keeping the diffusion model frozen. Experiments on real-user preference data show that our approach achieves approximately 77% held-out pairwise preference prediction accuracy and improves the alignment of generated images with individual user preferences.