Text to image models wrongly show figurative vehicles as objects
Presence Is Not Faithfulness: Figurative Vehicle Intrusion in Text-to-Image Generation
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Text-to-image models try to turn words into pictures, but they sometimes get confused with figurative language. For example, if a phrase uses a vehicle as a metaphor, the model might draw that vehicle literally instead of capturing the intended meaning. The authors call this problem Figurative Vehicle Intrusion. They created a benchmark called VISTA to measure this issue and introduced a score called V-Score to evaluate how well models handle it. They also proposed a method called VISTA-Guard that helps reduce this problem in generated images.
What this means in practice
- •For graphic designers: Improve text-to-image tools to generate images that better capture figurative meanings in prompts.
- •For multilingual ai developers: Use VISTA to evaluate and reduce errors related to figurative language in text-to-image systems across multiple languages.
Authors
Xiaoyu Ma, Chen Yang, Hao Chen
Abstract
Text-to-image (TTI) models increasingly generate high-quality images from natural-language prompts, yet figurative language exposes a failure: a vehicle that should guide the depiction of a tenor may instead be rendered as a visible object. We call this failure Figurative Vehicle Intrusion: the intruding content is textually licensed, but it is assigned the wrong visual role, showing that visual presence is not always faithfulness and that presence-oriented evaluation can miss such errors. To study it systematically, we introduce Vehicle Intrusion and Semantic Tenor Assessment (VISTA), a multilingual benchmark of figurative prompts organized by Figurative Form and Mapping Mechanism. We further propose V-Score, a diagnostic question-answering metric that evaluates role-aware figurative faithfulness in generated images. Evaluations on recent high-performing TTI models show that vehicle intrusion persists across languages and figurative categories. As a lightweight mitigation, we introduce VISTA-Guard, which partially reduces vehicle intrusion and suggests a practical path toward more figuratively faithful TTI generation. All resources will be released publicly.