The Illusion of High Utility in Safety Alignment of Text-to-Image Diffusion Models
2026-07-01 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial IntelligenceMachine Learning
AI summaryⓘ
The authors studied how text-to-image models can be made safer by preventing harmful images but found that common tests miss detailed mistakes in generated pictures. Using a specialized test called TIFA, they discovered that safety measures often reduce the accuracy of small details like object counts and relationships. They traced this problem to changes in how the model understands text prompts, which they call semantic collapse. To fix this, the authors proposed a new method called SAGE that keeps the model's understanding of prompts more structured, leading to better detail accuracy while still being safe.
text-to-image diffusion modelssafety alignmentsemantic fidelityTIFA evaluationtext-encoder embeddingssemantic collapseembedding spacegeometric regularizationstructured utilityprompt embedding
Authors
Adeel Yousaf, Soumik Ghosh, James Beetham, Amrit Singh Bedi, Mubarak Shah
Abstract
Safety alignment of text-to-image (T2I) diffusion models aims to suppress harmful generations while preserving utility on benign prompts. Recent methods often appear to deliver high safety with high utility, but this conclusion rests largely on coarse global utility metrics (e.g., FID, CLIPScore) that are insensitive to fine-grained semantic correctness, creating an illusion of high utility. We show that when utility is measured with structured evaluation, this illusion breaks: on TIFA (Text-to-Image Faithfulness evaluation with Question Answering), safety-aligned models suffer substantial drops in semantic fidelity, including failures in object counts, attributes, and relationships. To diagnose the source of this gap, we analyze the text-encoder prompt embedding space and uncover semantic collapse, a contraction of embedding spread coupled with distortion of inter-prompt similarity structure, which strongly correlates with structured utility loss. Guided by this insight, we propose StructureAware Geometric Regularization (SAGE), a safety alignment objective that explicitly preserves embedding spread and inter-prompt relational structure during adaptation. Our method restores structured utility (TIFA +5.0% over prior state-of-the-art) while maintaining strong safety performance and competitive coarse-grained utility scores. Our source code and trained models are available at https://adeelyousaf.github.io/SAGE_ECCV26_Project_Page/.