Conditional fisher information approaches gaussian limits under log concavity
Conditional Fisher-Information Central Limit Theorems under Log-Concavity with Information-Theoretic Consequences
Information Theory
Summary
This paper explores how certain mathematical measures related to randomness and information change when many small, similar random variables are added together under special smooth conditions called log-concavity. The authors show that a measure called Fisher information behaves predictably and approaches that of a bell curve (Gaussian), which is a common model for randomness. They also connect these results to how information and uncertainty behave in communication systems with noise. This helps to understand limits on transmitting information efficiently when the inputs or signals have fixed characteristics.
Fisher informationcentral limit theoremlog-concavityconditional covarianceGaussian distributionFisher information matrixrelative entropymutual informationGaussian noiselogarithmic Sobolev inequality
Authors
Tong Ye, Liu-Quan Yao
Abstract
We establish conditional central limit theorems in Fisher information under log-concavity in every fixed dimension. For conditionally centered normalized sums, after whitening by the averaged conditional covariance, the averaged conditional Fisher information converges to the dimension if and only if it is finite at one convolution level. The scalar criterion follows as the one-dimensional case; we also provide an independent scalar proof based on a second-order continuity theorem for Fisher production on Gaussian-smoothed, tail-controlled classes. For the original sums, the averaged Fisher information matrix converges in operator norm to the inverse averaged conditional covariance. Consequently, the conditional relative Fisher information with respect to the limiting Gaussian law vanishes, and the Gaussian logarithmic Sobolev inequality yields convergence in conditional relative entropy and conditional entropy. We give two operational consequences. For any fixed finite-constellation low-power input, the first-order conditional mutual-information slope converges to the Gaussian-noise benchmark. For Gaussian signaling at any fixed signal covariance, the mutual-information gap from that benchmark is bounded by the conditional relative Fisher deficit and hence vanishes asymptotically.