Discriminative models reveal a new way to generate and edit images
Discriminative Score Function: Turning Pretrained Models into Functional Generative Priors
Computer Vision and Pattern RecognitionMachine Learning
Summary
Deep learning models trained to recognize images also contain hidden clues about how real images look. The authors show a method to turn these clues into a tool that can create new images, change parts of images, or fill gaps without needing extra training. This tool works with various popular pretrained models and helps adjust systems without needing real data or labels. Their approach uncovers a surprising connection between models built for understanding images and the ability to generate them.
What this means in practice
- •For computer vision engineers: Create and edit images using existing pretrained models without retraining or needing labeled data, simplifying image generation workflows.
- •For machine learning operations teams: Improve zero-shot calibration of image recognition systems with synthetic images generated from pretrained models when real data is unavailable.
Authors
Junhoo Lee, Hyeonjin Kim, Sangbum Han, Nojun Kwak
Abstract
Discriminative models are trained with task-specific objectives such as classification, detection, or representation learning, yet their practical utility depends on generalizing beyond the training task to unseen samples from the same domain. This suggests that a discriminative model may retain an implicit trace of the data manifold beyond its task labels. We ask whether this trace can be exposed as a generative signal without task-specific guidance. We introduce the Discriminative Score Function (DSF), which extracts an image-space generative field from the loss-gradient geometry of a pretrained discriminative model. DSF provides a training-free update rule that moves images toward model-consistent visual regions without retraining or task-specific targets. We apply DSF to diverse pretrained discriminative models, including ResNet classifiers, DETR detectors, and DINOv2 ViT encoders. DSF generates unconditional images from noise and supports conditional guidance, editing, inpainting, and explanation through the same field. DSF-generated samples also provide effective zero-shot quantization calibration when real images or labels are unavailable, preserving feature geometry better than data-free calibration baselines. These results show that discriminative training leaves a generative functional signal for synthesis and calibration. Project page: https://junhoo.me/dsf.