Evaluating the Representation Space of Diffusion Models via Self-Supervised Principles
2026-06-08 • Machine Learning
Machine LearningComputer Vision and Pattern Recognition
AI summaryⓘ
The authors study diffusion models, which can create new data and also learn useful patterns without labels. They introduce a method to separate the stable parts of learned features from the variable parts, measuring this with something called the Invariant Contamination Ratio (ICR). They find that the best stable features appear at moderate noise levels and relate to better classification results. They also use ICR to detect when the model starts memorizing the training data instead of generalizing, just by looking at its features during training. This work provides a way to understand diffusion models through the patterns they learn, without extra tests.
Diffusion modelsSelf-supervised learningInvariant featuresResidual featuresInvariant Contamination RatioFisher informationGenerative modelsRepresentation learningMemorizationGeneralization
Authors
Xiao Li, Yixuan Jia, Zekai Zhang, Xiang Li, Lianghe Shi, Jinxin Zhou, Zhihui Zhu, Liyue Shen, Qing Qu
Abstract
Diffusion models have demonstrated remarkable generative capabilities and have also emerged as powerful self-supervised representation learners, yet the connection between these two abilities remains less explored. Drawing inspiration from self-supervised learning (SSL), we introduce a framework for jointly evaluating the representation and generation capabilities of diffusion models. Specifically, we decompose features into invariant and residual components and derive the Invariant Contamination Ratio (ICR), a Fisher-based metric that quantifies how residual variation contaminates invariant signal in feature space. We use this framework to analyze both discriminative and generative behavior of diffusion models. On the representation side, we find that invariance peaks at intermediate noise levels, which also yield the best downstream classification performance. On the generative side, we study how training transitions from genuine generalization to memorization in data-limited regimes, and show that ICR serves as a sensitive training-time indicator of early learning: increasing residual energy along Fisher directions marks the onset of memorization, detectable from training features alone without external evaluators or held-out test sets. Overall, our results show that diffusion models can be monitored from a self-supervised perspective through the geometry of their learned representations.