Rao-Blackwellized Score Matching on Manifolds
2026-05-25 • Machine Learning
Machine Learning
AI summaryⓘ
The authors study a method called denoising score matching (DSM) when the data lies on a curved surface (a manifold) inside a higher-dimensional space. They find that the usual noise model creates tricky infinite variance directions, but by focusing on a special projection onto the manifold, this problem is fixed in an optimal way. They then analyze how the ideal denoising target behaves for small noise and show it matches a known geometric score with some extra terms that depend on the manifold's shape. Their results simplify nicely in flat spaces and on spheres, revealing interesting curvature-related corrections.
denoising score matchingmanifoldtangent spaceGaussian noiseRao-BlackwellizationRiemannian scoreTweedie formulaWeingarten operatorRicci curvaturesmall-noise expansion
Authors
Divit Rawal
Abstract
We study denoising score matching (DSM) when the latent distribution is supported on a smooth embedded manifold $M \subset \mathbb{R}^D$. Under ambient Gaussian corruption, the tangent denoising target contains a singular normal-fiber noise channel whose variance diverges as $d/σ^2$ as $σ\to 0^+$. We show that conditioning on the nearest-point projection $π(X)$ canonically removes this singularity: the resulting conditional expectation is the unique $L^2$-optimal Rao-Blackwellized predictor of the tangent DSM target among all estimators depending only on the projected observation $π(X)$. We then compute the small-noise expansion of this canonical target and show that it equals the intrinsic Riemannian score up to an explicit order-$σ^2$ correction that decomposes into an intrinsic Tweedie term and an extrinsic curvature term involving the Weingarten and Ricci operators. In the flat case, the construction reduces exactly to ordinary lower-dimensional Gaussian DSM, while on $S^d$ the extrinsic correction simplifies to the scalar factor $(1-d/2)\nabla_M \log q$; this extrinsic $σ^2$ correction cancels identically on $S^2$, though the intrinsic Tweedie term remains.