Image verification improves by checking facts beyond visual cues
WorldFact-Bench: Beyond Image-Internal Plausibility to Image-World Consistency
Computer Vision and Pattern Recognition
Summary
Images generated by AI can look very real, but sometimes they show things that don’t match facts from the real world. The authors created a test called WorldFact-Bench to check if an image is not only believable but also factually correct about the world. They also made a system called PERSIST-Agent that keeps track of different facts and evidence during the checking process to better decide if an image matches reality. This system performed better than other methods, showing it's helpful to remember and verify facts step-by-step rather than just looking at the image by itself.
What this means in practice
- •For digital content moderators: Detect images whose appearances are plausible but contain real-world factual errors to improve content verification accuracy.
- •For news verification teams: Identify manipulated images that contradict known facts, supporting fact-checking efforts with more reliable tools.
Authors
Zhuohong Chen, Zhengxian Wu, Yunyao Yu, Hangrui Xu, Zijian Yu, Hao Tan, Zhifang Liu, Peng Jiao, Jun Lan, Haoqian Wang
Abstract
Advances in image generation have made visual authenticity increasingly difficult to assess. Although image forensics now examines both generation artifacts and higher-level visual inconsistencies, a plausible image can still contradict real-world facts or rules. We introduce WorldFact-Bench to evaluate image-world consistency from a single image, without a predefined claim or verification target. The benchmark contains 1,274 source-aligned real-fake pairs across four verification regimes and ten semantic domains. Each pair introduces a specific, evidence-supported factual conflict while seeking to preserve non-target content and visual plausibility. Images are evaluated independently, and pair accuracy requires both members of a pair to be classified correctly. We further propose PERSIST-Agent, which organizes iterative verification around a persistent state linking candidate facts, visual observations, evidence, and verification statuses. This state guides subsequent inspection and retrieval while retaining unresolved candidates. With backbone weights fixed, harness self-optimization refines the agent's prompts and execution rules through validation feedback. Experiments reveal strong label biases in several detectors and uneven gains from retrieval. On the evaluated 8B backbones, PERSIST-Agent improves pair accuracy over both direct judgment and retrieval-augmented baselines, while ablations support the role of persistent verification state. These findings highlight the value of state-guided verification and the remaining gap between visual plausibility and factual correctness.