Denoising Models Develop Human-Like Perceptual Illusion Representations Across Architectures
Computer Vision and Pattern Recognition
Summary
The authors studied how deep neural networks process brightness illusions and found that certain internal parts of these networks react to illusions, even if the final outputs don't show obvious changes. They discovered that this illusion sensitivity is influenced more by how the networks are trained (denoising) than by their structure. These internal reactions also closely match human brightness perception models and change predictably with illusion strength. Importantly, when the authors removed these illusion-sensitive parts, it affected the network’s internal signals, but changing these signals didn’t visibly affect the output images. They call these hidden reactions "perceptual phantoms" because they exist inside the model but can't be seen outside.
Authors
Gautam Ranka, Paras Chopra
Abstract
Deep neural networks trained on natural images are shown to produce outputs consistent with human observers for brightness illusions. While this phenomenon has been documented across architectures, all evidence, to date, is measured at the output level: restored pixels, decoded trajectories, or classification decisions. Whether these models actually represent illusions internally, and if so where and how, remains unknown. We show that denoising models develop illusion-sensitive representations at specific internal layers, across varied architectures. Specifically, we identify the layers and channels that discriminate illusory from physically matched control regions. We show that the denoising objective is a more important driver of the effect than the architecture. On domain-appropriate stimuli, these activations track a validated psychophysical model of human brightness perception (FLODOG; Spearman $ρ\geq 0.70$) and scale monotonically with parametric illusion strength. Leveraging these findings, we provide causal evidence via channel ablation showing that illusion-sensitive channels specifically and substantially affect the internal signal. Yet injecting these representations into the generation pipeline produces no measurable pixel shift across all tested architectures; we term such representations perceptual phantoms: active in internal processing yet invisible to any output-based evaluation. While related internal-output dissociations have been characterized in language models, this is the first such characterization for perceptual representations in denoising vision models.