One-step lowest-variance selection in a Gaussian random-field model motivated by masked diffusion: Total correlation and a square root collision threshold
2026-07-20 • Machine Learning
Machine LearningArtificial IntelligenceInformation Theory
AI summaryⓘ
The authors study a simplified mathematical model to understand how picking multiple low-uncertainty positions at once affects their dependence when decoding data in parallel. They use a Gaussian random field to represent uncertainty scores and look at how selecting the lowest scores in a connected area influences the correlation between those positions. Their results show that if the selection is small, dependence disappears, but if it grows at a certain rate, dependence remains significant. The study offers a clear baseline to analyze how selection size, score relationships, and spatial layout impact confidence-guided decoding methods.
Gaussian random fieldmasked discrete diffusionparallel decodinguncertainty scoresspatial correlationtotal correlationstochastic geometryconfidence-based selectionasymptotic probability
Authors
Linjun Li
Abstract
Motivated by confidence-guided parallel unmasking in masked discrete diffusion, we study a single selection step in a stylized Gaussian random-field model. A locally dependent nonnegative score field represents position wise uncertainty, and the scheduler selects the K positions with the smallest scores. Dependence among the selected positions is measured through a distance-dependent Gaussian correlation model. This separation provides a tractable framework for quantifying how the geometry of low-score locations affects the dependence cost of factorized parallel decoding. We establish two complementary results. In a conservative sub-square-root regime, the conditional Gaussian total correlation of the selected block vanishes in probability. At the square-root scale, it remains non-negligible with positive asymptotic probability and admits a strictly positive expectation lower bound. Synthetic experiments support the predicted finite-size behavior. These results provide a rigorous stochastic-geometry baseline for understanding how budget size, score dependence, and spatial correlation jointly shape one-step confidence-based selection in masked discrete diffusion.