Diffusion model optimization often misses true human intent
When Optimization Meets Human Intent: Human-Centered Diffusion Model Optimization
Computer Vision and Pattern Recognition
Summary
Diffusion models are tools that create images and other content, but improving them usually focuses on easy-to-measure things like looks. The paper points out that such improvements don’t always match what people really want, because human needs are complex and change depending on context. The authors analyze how human intentions are captured, turned into goals for the model, and measured through outputs. They highlight gaps between automated scores and actual user satisfaction in personalized image generation.
What this means in practice
- •For personalization engineers: Create image generation tools that better balance human intent against automated quality metrics for improved user satisfaction.
- •For user experience designers: Design evaluation processes that include human feedback to complement automated scores when refining generative models.
A position paper. It proposes an approach and reports no results.
Authors
Renye Yan, Jikang Cheng
Abstract
Diffusion models are evolving from general tools into personalized systems that adapt to user needs through fine-tuning and guidance. However, diffusion optimization mainly targets measurable objectives such as aesthetics, rarely examining whether they align with human intent. Human needs are multidimensional, contextual, and dynamic, whereas existing methods approximate them with fixed metrics. Because these metrics capture human needs only partially and may amplify deviations from true intent during optimization, higher scores need not yield results better matching human expectations. To understand how human needs enter and influence optimization, we systematically analyze how human intent is expressed, translated into optimizable objectives, applied to model behavior or generation, and evaluated through outputs. We further discuss trade-offs affecting human-intent alignment in personalized image generation and gaps between automated metrics and real user judgments. This human-centered synthesis reorganizes diffusion optimization and provides a systematic reference for personalized generation systems designed for human preferences.