One Inverse Step is a Convex Program: Bayes-Limit Calibration of Diffusion Inversion
2026-08-24 • Machine Learning
Machine Learning
AI summaryⓘ
The authors study a mathematical step in diffusion models, showing it relates to how these models capture local data structure without needing assumptions about the data shape. They find conditions where the solution is unique and identify when the model’s score estimates fail, providing a way to detect errors in the model. They also show that the iteration method used can be unstable unless adjusted, and that current trained models do not exhibit certain predicted geometric features due to limitations in training. Finally, they reveal a theoretical bound is violated in popular diffusion model settings, highlighting a gap between theory and practice.
DDIM inversiondiffusion modelsscore functionBayes limitposterior covariancePicard iterationHessianlog-SNRDDPM schedulemodel error certificate
Authors
Gordei Verbii
Abstract
One implicit DDIM inversion step is the cheapest probe of whether a pretrained diffusion model encodes local manifold geometry. It is the stationarity condition of an explicit potential, $x-G(x)=\nablaΨ_t(x)$, strongly convex at the Bayes limit with modulus exactly $e^{-h_t}$ for the step's log-SNR gap $h_t$ $-$ for every data law, schedule and point, with no manifold, reach or unimodality hypothesis. Three consequences must be kept apart. (i) The solution is unique at the Bayes limit; a second one requires the trained score to violate the posterior-covariance bound by $1/(1-e^{-h_t})$, a hypothesis-free certificate of model error; the same bound makes contraction a schedule constant, $ρ_g^{\star}=1-e^{-h_t}<0.326$ throughout the standard DDPM schedule. (ii) The solver can still fail: Picard iteration is unit-step gradient descent on $Ψ_t$, unstable wherever $λ_{\max}(\nabla^2Ψ_t)>2$, so oscillation certifies nothing; damping below $2/λ_{\max}$ cures it. (iii) The geometry lives in the convergence domain: on the scale-free depth $w=rκ_{\max}$ the oscillation shell sits at $w=\tfrac12$, schedule-free, and the divergence shell at $w=1/(1+ρ_g^{\star})$, with a measured finite-noise correction in $\|\mathrm{II}\|^2$. Exact scores reproduce both to within $0.54\%$ on three classes; no trained score we probe shows a shell $-$ a derived limitation, not a null result: the Fermi window conflicts with the model's own training support by $3.6$-$5.6\times$, and the trained Hessian-Lipschitz constant is $2$-$12\%$ of the curvature the law reads, $0$ on a ReLU net. Finally the unconditional ceiling $σ_tλ_{\max}(\mathrm{sym}\,J)\le1$, from $\mathrm{Cov}(x_0\mid x_t)\succeq0$ alone, holds for the exact score to $3\times10^{-7}$ but is violated in all DDPM CIFAR-10/CelebA-HQ-256 settings, by $1.26$-$4.66\times$.