Gaussian distribution uniquely makes estimation error smoothly expand at low noise
Zero-SNR Analyticity of the Scalar MMSE Is Equivalent to Gaussianity
Information TheoryMachine Learning
Summary
This paper looks at how accurately one can estimate a hidden variable when it is observed in noisy data. The authors find that if the variable isn’t Gaussian (a common bell-curve shape), the error in estimation behaves in a complicated way as noise gets very low. They prove that only a Gaussian variable leads to a smooth, simple expansion of estimation error at very low noise. This connection helps understand the special role of Gaussian distributions in signal processing and information theory.
What this means in practice
- •For signal processing engineers: Design estimation algorithms knowing that Gaussian inputs uniquely allow smooth low-noise approximations of estimation error.
- •For communication system designers: Use the identified analytic characterization to assess channel input assumptions affecting signal decoding accuracy near zero signal-to-noise ratios.
A theory result. No direct application yet.
Authors
Yixing Zhang
Abstract
Let $Y_s=\sqrt{s}X+Z$, where $Z$ is standard Gaussian and independent of the real random variable $X$. We prove that, under the square-exponential moment condition $\mathbb{E}e^{βX^2}<\infty$ for some $β>0$, the scalar minimum mean-square error $\operatorname{mmse}_X(s)$ is analytic at zero signal-to-noise ratio if and only if $X$ is Gaussian, with constant random variables included as degenerate Gaussians. The proof converts estimation in the Gaussian channel into a backward heat flow acting on the moment-generating function $M(z)=\mathbb{E}e^{zX}$. Under the stated tail condition, every non-Gaussian input forces $M$ to have a nonzero complex zero. We show that each zero cluster produces a finite singularity in its localized Borel transform at the action $ξ=z_0^2/2$. After removing the action scale, the Borel coefficients have a nonzero $n^{-1/2}$ prefactor for a simple zero. A zero of multiplicity $m\geq 2$ splits according to the roots of a Hermite polynomial and instead contributes a prefactor $n^{-m/2}e^{r_m\sqrt{2n}}$. A finite-disc localization and relative-cycle continuation argument then show that at least one such singularity survives in the full Borel transform. Thus, for every non-Gaussian input in the stated class, the formal zero-SNR expansion is Gevrey-1 but divergent. Rational-MMSE rigidity and the analogous analyticity criterion for mutual information follow as corollaries.