Ufo evaluates multi-condition alignment in image generation models
UFO: Chain-of-Evaluation for Omni-Condition Alignment in Multi-Modal Image Generation
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Multi-modal image generation models try to create images based on several different input types, like text and images, all at once. But existing ways to check if these images match all inputs at the same time aren't very good and don’t agree well with what people think. The authors came up with a new method called UFO that breaks the checking process into small parts focused on each type of input, then combines the results for a better overall judgment. Their tests show that UFO matches human opinions much better than older methods. They also created a benchmark to test how well different image generators handle multiple input conditions together.
What this means in practice
- •For ai product developers: Improve quality control in AI-based image generation apps by evaluating alignment with multiple input types simultaneously.
- •For benchmarking teams: Use UFO-Bench to measure and compare how well customization models handle combined text and image inputs.
Authors
Danning Zhang, Yijing Lin, Shuhan Zhuang, Mengqi Huang, Shaojin Wu, Shancheng Fang, Zhendong Mao
Abstract
Multi-modal image generation, particularly subject-driven customization, has garnered growing attention in recent years. Despite the rapid advancement of generative models, their evaluation remains largely lagging. Existing methods, whether embedding-based or Multi-modal Large Language Model (MLLM)-based, evaluate alignment with each modal condition in isolation, which contradicts the simultaneous condition alignment objective of multi-modal image generation, leading to poor consistency with human judgments. To address this challenge, we propose UFO, the first unified framework for omni-condition alignment simultaneous evaluation. Specifically, UFO introduces a novel Atomized Chain-of-Evaluation paradigm, \emph{i.e.}, it first decomposes omni-condition alignment into a sequential chain of fine-grained, disentangled Atomic Evaluation Units (AEUs), categorizes them into distinct modality-relevance classes, and then employs general or dedicated functional calls for accurate verification of different AEU types. Experimental results demonstrate that UFO achieves the highest correlation with human evaluation preferences, delivering an average improvement of 15.25\%. Furthermore, we present UFO-Bench, a dedicated benchmark designed to holistically evaluate the performance of existing customization models under the diverse mutual interactions of textual and visual conditions.