Method improves emotional image generation by preserving image content
Semantic-First MixGRPO: Reliable Emotional Image Generation via Temporal Decoupling
Computer Vision and Pattern Recognition
Summary
Changing the emotional tone of an image without altering the objects or layout can be difficult. The authors explore a new way to control emotional expression during image generation by separating the process into two phases: first establishing the image content, then adjusting its emotional tone. This approach keeps the main subjects in the images stable while reliably changing their emotional appearance. It also reduces training costs compared to some other methods, though it involves a trade-off between emotional accuracy and content preservation. A study with human participants found this method produces preferred images for emotional consistency.
What this means in practice
- •For digital artists: Create images with targeted emotional expressions while keeping the main content intact for design and storytelling.
- •For game developers: Generate emotionally varied character or scene visuals on demand without altering key objects or layout.
Authors
Zhenxuan Wang
Abstract
Controllable image synthesis should modify visual affect without replacing the objects, layout, or relations specified by a prompt. Existing methods address this tension through conditioning architectures or auxiliary losses, while mixed ordinary differential equation (ODE) and stochastic differential equation (SDE) sampling has mainly been used to accelerate group relative policy optimization (GRPO). We instead study stochastic-window position as an attribute-control variable for flow-based image generation. Using FLUX.1-dev with EmotiCrafter-style valence-arousal conditioning, we compare fixed early, middle, and late four-transition SDE windows, progressive MixGRPO, and full-path Flow-GRPO under matched settings. Because step 0 has the highest noise, our semantic-first schedule uses deterministic ODE sampling at steps 0-19 to establish content, then confines stochastic exploration and policy updates to steps 20-23. Across three seeds, it achieves a CLIPScore of $29.662\pm0.276$, cross-affect DINO consistency of $80.1\pm0.9$, and semantic drift of 2.9%, while reducing training cost by 33.5% relative to full-path Flow-GRPO. Full-path stochastic training remains more accurate on valence-arousal targets, revealing a genuine trade-off. Semantic-first nevertheless improves both affect and semantic metrics over progressive scheduling, preserves the principal subject in 95.98% of images at five extreme or neutral targets, and receives the highest semantic-preservation and overall-preference ratings in a 30-participant study. These results identify probability-flow time as a practical graphics control dimension for reliable affective appearance manipulation.