Saliency maps reveal unique flaws in AI generated images
Can We Use Saliency Maps for the Evaluation of Generative Image Models?
Computer Vision and Pattern Recognition
Summary
It is hard to judge how good AI-generated images look because usual tests rely on other AI tools trained for different tasks. The authors found a way to test these images better by training a special AI to tell real images from fake ones made by a specific model. This AI then shows which parts of the image helped it decide, highlighting unique flaws each image generator has. Combining the AI’s accuracy and these highlighted parts gives a score that matches human opinions better and doesn’t need big pre-trained AI models.
What this means in practice
- •For ai model developers: Assess the quality of their specific image generators by identifying model-specific visual flaws without relying on unrelated pretrained networks.
- •For digital forensics teams: Detect image forgeries by using classifier-driven saliency maps that expose unique artifacts of different image synthesis models.
Authors
Eleonora Ristori, Luca Bindini, Marco Lippi, Paolo Frasconi
Abstract
Evaluating generative image models remains challenging, as commonly used metrics depend on fixed pretrained feature extractors optimized for different tasks that do not always align with human realism perception across datasets and model families. To address this limitation, we investigate a human-inspired evaluation framework that avoids representations learned for unrelated tasks. Specifically, we train a generator-specific classifier to distinguish real images from samples produced by a given model and analyze its decisions using saliency maps. Across diverse benchmarks, we observe that these classifiers learn generator-specific artifacts rather than a universal notion of "fakeness", exhibiting strong accuracy and limited cross-generator transfer. This makes saliency analysis particularly informative: the resulting attribution maps highlight visual cues and structural irregularities characteristic of each model's outputs. By combining classification accuracy and saliency information, we obtain a composite score aligned with human judgments, without relying on large pretrained backbones and using only real and generated samples from the evaluated model.