TerraVis measures real-world visual errors in AI image generation
TerraVis: Towards Evaluation of World-Grounded Visual Consistency in Text-to-Image Generation via MLLM Workflows
Computer Vision and Pattern Recognition
Summary
Text-to-image AI can create beautiful pictures, but sometimes these pictures include impossible scenes or objects that don’t make sense in the real world. The authors developed TerraVis, a new way to check if AI-generated images follow real-world rules about how objects look and interact. TerraVis uses a detailed checklist and a large AI model to find and score these errors, helping to understand which image generators create more realistic scenes. They showed that TerraVis matches human opinions better than older methods and exposes problems that other scores miss.
What this means in practice
- •For visual content developers: Evaluate the real-world correctness of AI-generated images to improve reliability in visual media production.
- •For augmented reality engineers: Detect and reduce world-consistency errors in generated images to enhance realistic scene construction in AR applications.
Authors
Shuai Fu, Jing Gu, Jian Zhou, Zicheng Duan, Gengze Zhou, Qi Wu
Abstract
Recent text-to-image models have made substantial progress in photorealism, aesthetics, and text-image alignment. Yet visually appealing images can still violate real-world plausibility, exhibiting malformed object structures, impossible anatomy, physically implausible interactions, or inconsistent spatial relationships. Such failures are not well captured by existing fidelity, aesthetics, preference, or alignment metrics. To address this gap, we introduce TerraVis, a framework for evaluating world-grounded visual consistency in generated images. TerraVis defines a structured taxonomy of world-consistency violations spanning object-, interaction-, and scene-level failures, and employs a multi-stage evaluation framework to identify and quantify them. Given an image, TerraVis first uses an MLLM to assess its eligibility for evaluation, then detects violations across 18 taxonomy-defined types and classifies them as minor or major to derive an overall world-consistency score. Across diverse open-source and proprietary text-to-image models on two widely used benchmarks, TerraVis achieves the strongest correlation with human judgments of world consistency among existing metrics. Our benchmark results further show that models that achieve strong performance on conventional metrics can still exhibit substantial world-consistency failures. These findings highlight world consistency as a complementary evaluation dimension and demonstrate that TerraVis enables systematic quantification, diagnosis, and comparison of such failures. Our code is publicly available at https://github.com/ShyFoo/TerraVis.