Sharp mutual information limit found for binary variable pairs
A Sharp Mutual Information Bound for Binary Pairs
Information Theory
Summary
This paper finds the exact maximum amount of shared information two yes-or-no (binary) variables can have, based on a specific measurement of their relationship. The authors prove a mathematical formula that matches the best-possible bound for this case, solving a problem posed by earlier researchers. They show when this bound is exactly met: either the variables are completely unrelated or structured in a very balanced way. This result also helps understand information limits when dealing with sequences of related binary observations.
What this means in practice
- •For communication engineers: Determine precise limits on information transmission efficiency for binary signaling systems under noise constraints.
- •For signal processing developers: Design better binary-output filters by understanding how correlated noisy inputs limit achievable mutual information.
A theory result. No direct application yet.
Authors
Qianmian Guo
Abstract
We prove a sharp upper bound on the mutual information between two binary random variables in terms of their distribution-dependent KL contraction coefficient. This resolves the binary-pair conjecture of Anantharam, Gohari, Kamath, and Nair. Equality holds if and only if the variables are independent or form a doubly symmetric binary source. The proof uses supporting tangents of an entropy curve to vary the input distribution and the channel while preserving the contraction coefficient. Together with tensorization and data processing, the result also bounds the mutual information between binary outputs obtained separately from two correlated observation sequences. When the observation pairs are independent copies of a finite-alphabet pair, this bound depends only on the law of one pair and not on the number of observations.