Diffusion model attribution reveals how training examples shape images
Attributing HOW, Not Just WHICH: Counterfactual Response Trajectories for Diffusion Models
Computer Vision and Pattern RecognitionGraphics
Summary
Diffusion models create images by gradually improving noisy pictures, but it's hard to tell how specific training images influence the final result. The authors focus not just on which training examples matter, but on how their impact changes during the image-building process. They introduce a method called CADT that tracks step-by-step changes inside the model, comparing them to similar changes from training data. This approach helps to better understand how different parts of the model and training examples contribute to image generation.
What this means in practice
- •For ai model developers: Identify how and when specific training images influence output during diffusion-based image generation to improve model interpretability and debugging.
- •For digital artists: Gain insights into which training image features shape generated artworks over time, enabling more precise style and content control.
Authors
Haoqian Zhang, Ziyuan Yang, Zerui Shao, Yi Zhang
Abstract
Diffusion models have achieved remarkable success in image generation, yet tracing their outputs to individual training examples remains challenging. Existing attribution methods often compress factor-specific effects into scalar responses, making distinct internal changes indistinguishable. This is particularly limiting for diffusion models, where semantic factors emerge through evolving representation dynamics during denoising. We therefore reformulate diffusion data attribution as attributing factor-induced internal response trajectories. In this paper, we propose a novel Concept Attribution method through Dynamic Trajectories(CADT). We argue that attribution should therefore ask not only \emph{which} examples matter, but also \emph{how} their influence unfolds during generation. Specifically, we construct matched counterfactual pairs at identical noisy states to isolate factor-specific representation displacements, and model their directional and magnitude evolution across denoising as dynamic attribution signatures. For each training example and generated query, CADT extracts stage-wise feature vectors and integrates them along the denoising process to form a trajectory descriptor. Applying the same construction across the training set yields a bank of factor-specific trajectory descriptors. The covariance statistics of this bank are then used to construct . CADT uses this covariance-aware positive-semidefinite kernel to calibrate the query and training representations, and compares the calibrated query trajectory with each training trajectory to produce the final training-sample attribution scores. Experiments on multiple public datasets show consistent improvements over existing diffusion attribution baselines across hierarchical, compositional, and style attribution.