Diffusion models controlled to delay copying and improve sample quality
Controlling Polar Exposure to Delay Memorization in Diffusion Models
Machine Learning
Summary
Diffusion models can sometimes copy their training pictures too soon, which limits how well they learn to create new images. The authors studied how the training updates affect this copying and designed a controller called Quality-Gated De-whitening (QGD) to manage when the model should slow copying and focus on quality improvement. They also created a method called Copy-Budgeted Selection (CBS) to better choose the best model versions. Their approach delays copying by over eight times while keeping good image quality and reduces copying by almost 76%. This method helps make generated images better without memorizing the training data too early.
What this means in practice
- •For machine learning engineers: Improve generative image model training by delaying early copying of training data, extending when models produce novel samples of high quality.
- •For digital artists and content creators: Generate higher-quality images with less risk of replicating training images too soon, improving originality in creative outputs.
Authors
Xuanchen Wang, Heng Wang, Weidong Cai
Abstract
Diffusion models can reach useful sample quality before copying training examples, but fast optimization can compress this generalization window by accelerating sample-specific fitting. We investigate this effect through update geometry and propose Quality-Gated De-whitening (QGD), a controller that retains a fast polar-update prefix and progressively restores fixed-gain momentum. Our random-feature analysis separates covariance-controlled, curvature-equalized and amplitude-controlled memorization clocks. Under aligned spectral assumptions, it establishes a finite-exposure condition under which a fixed-gain tail recovers a delay proportional to dataset size. QGD implements this principle with a confirmed quality gate, a bounded decay envelope and causal copy feedback. Immediate switching is the conservative limit; gradual control balances delayed copying against continued quality improvement. We pair QGD with Copy-Budgeted Selection (CBS), which applies simultaneous binomial calibration to a frozen checkpoint family, followed by a fresh evaluation of the released checkpoint. On 2,000-image CIFAR-10 subsets, QGD preserves the polar baseline's quality-arrival time while expanding its useful interval by 8.32x and reducing common-checkpoint copying by 75.9%. With identical calibration and independent quality evaluation, QGD achieves FID 75.56 versus 79.37 for SGD with the same selector. Exposure-matched controls, independent detector audits and transfer to flow matching and dance generation support adaptive exposure control as a practical way to improve the quality-copying tradeoff.