Conditional fisher information limits clarify gaussian behavior in noisy systems

Conditional Fisher-Information Central Limit Theorems under Log-Concavity with Information-Theoretic Consequences

Information Theory

Summary

This paper studies how certain mathematical measures, called Fisher information, behave when adding up random effects that have a special shape known as log-concavity. The authors prove that the measure approaches a simple, predictable form related to Gaussian noise in many dimensions. This helps show that some technical quantities used in information theory become simpler and behave like Gaussian noise under certain conditions. They also show practical consequences for how much information can be transmitted in noisy communication systems when using fixed sets of signals or Gaussian signals.

What this means in practice

  • For communication system designers: Estimate mutual information limits for low-power signaling using finite sets under Gaussian noise, ensuring performance nears Gaussian benchmarks.
  • For signal processing engineers: Bound the gap in mutual information achievable by Gaussian signaling through conditional Fisher information deficits to optimize system design.

A theory result. No direct application yet.

Authors

Tong Ye, Liu-Quan Yao

Abstract

We establish conditional central limit theorems in Fisher information under log-concavity in every fixed dimension. For conditionally centered normalized sums, after whitening by the averaged conditional covariance, the averaged conditional Fisher information converges to the dimension if and only if it is finite at one convolution level. The scalar criterion follows as the one-dimensional case; we also provide an independent scalar proof based on a second-order continuity theorem for Fisher production on Gaussian-smoothed, tail-controlled classes. For the original sums, the averaged Fisher information matrix converges in operator norm to the inverse averaged conditional covariance. Consequently, the conditional relative Fisher information with respect to the limiting Gaussian law vanishes, and the Gaussian logarithmic Sobolev inequality yields convergence in conditional relative entropy and conditional entropy. We give two operational consequences. For any fixed finite-constellation low-power input, the first-order conditional mutual-information slope converges to the Gaussian-noise benchmark. For Gaussian signaling at any fixed signal covariance, the mutual-information gap from that benchmark is bounded by the conditional relative Fisher deficit and hence vanishes asymptotically.
View PDFOpen arXiv