Generative models predict human sense of image naturalness
Do Generative Priors Align with Human Naturalness Perception?
Computer Vision and Pattern Recognition
Summary
This paper looks at whether computer models that create images can tell if images look natural the way people do. The authors tested 25 different image and video generators to see if their internal signals match human judgments about unusual changes, like weird facial expressions or odd lighting. They found that these models often agree with people about what looks strange or normal, even matching well-known effects like how upside-down faces look different to us. The study shows that these models capture human-like ways of seeing naturalness, beyond just spotting any error.
What this means in practice
- •For computer vision engineers: Improve image synthesis systems to better mimic human perceptions of naturalness by leveraging generative models' native error signals.
- •For quality assurance teams: Develop new automatic checks for image and video quality that align more closely with human judgments of unnaturalness using generative models’ loss patterns.
Authors
Taiki Fukiage
Abstract
Visual generative models are trained to capture the probability distributions of natural images, yet whether their native priors reflect the regularities governing human perception of image naturalness remains an open question. Here, we probe these priors through native prediction errors across 25 open image and video generators. Because raw single-image losses are dominated by scene content and visual complexity, we evaluate directional loss differences using content-preserving, paired relational interventions that selectively disrupt facial configurations or physical illumination consistency while limiting changes in low-level image statistics. Across both domains, these loss differences reproduce human-like selective sensitivities and tolerances, capturing the classic Thatcher effect on faces and shape-dependent responses to illumination inconsistencies. Notably, these loss differences reliably track continuous gradations of human naturalness judgments across individual stimulus pairs (peaking at $r = .84$ on faces and $.64$ on physical scenes) and retain unique human-aligned signals even after controlling for feature distances from frozen vision encoders and standard image quality metrics. We also find that while overall sensitivity to these violations broadly covaries with human alignment across models, the two systematically decouple along denoising schedules, with alignment peaking earlier than sensitivity, revealing that human-like naturalness judgments dissociate from generic violation detection. Together, these findings demonstrate that learning visual distributions yields generative loss landscapes that capture distinct aspects of human naturalness perception.