Personalized image generation using user histories beats baselines
Personalized Image Generation with Reasoning and Reflection
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Creating images that truly reflect a person's style and preferences is hard because people have rich histories online, including posts, images, and reviews. The authors made a new testing setup to measure how well AI can create images tailored to someone's unique tastes, using their entire online history. They introduced PEARL, a method that thinks and reflects in steps to produce images better matched to each person. PEARL did better than other methods in two tasks: placing products in user-specific scenes and creating new images that fit someone's visual style.
What this means in practice
- •For e-commerce teams: Produce product images set in scenes that reflect individual customer lifestyles, improving personalized product presentation online.$Commercial implications: Enables personalized product visuals tailored to users, enhancing shopping experience and boosting sales in online retail platforms.
- •For social media content creators: Generate novel images on specific topics that match a creator’s unique visual style and preferences for authentic social media posts.
Authors
Bo Ni, Ngoc N. Tran, Qinwen Ge, Franck Dernoncourt, Seunghyun Yoon, Samyadeep Basu, Sungchul Kim, Puneet Mathur, Nedim Lipka, Tong Yu, Yu Wang, Ryan A. Rossi, Tyler Derr
Abstract
Personalized image generation has remained narrowly focused on conditional synthesis from curated visual exemplars, rather than capturing who a user is. In practice, however, a user's personal context is much richer, comprising reviews, posts, images, captions, and metadata accumulated over time. A truly personalized generator should leverage this history to produce images aligned with the user's lifestyle and aesthetic preferences. To this end, we introduce the first unified benchmark for personalized image generation from user histories. The benchmark comprises two complementary tasks and a multi-axis evaluation protocol that assesses target fidelity, visual quality, user distinguishability, semantic alignment with the user's history, and task-specific utility. Grounded in real-world e-commerce and social media settings, the benchmark includes: (1) Personalized Scene Generation, which places a given object in a scene that reflects a user's preferences and lifestyle, motivated by personalized product presentation; and (2) Personalized Creative Generation, which generates a novel image on a specified topic that is faithful to a user's aesthetic and visual identity, motivated by social media content creation. We further propose PEARL, which couples a multimodal reasoner with a frozen image generator in an interleaved reason-reflect loop optimized with differential data reward. Across both tasks, PEARL outperforms strong baselines, achieving an average improvement of 15% across personalization metrics.