Binomial distribution has highest uncertainty among certain dependent sums
The binomial distribution maximises entropy among sums of negatively related indicators with fixed mean and support
Information Theory
Summary
This paper shows that when you combine multiple yes-or-no outcomes that are somehow negatively related, the most unpredictable total number you can get has a binomial distribution shape. The authors extend earlier work that proved this was true under a narrower set of conditions. They used some clever mathematical tools like Markov chains and probability orderings to prove their result. This helps us understand how randomness works when outcomes influence each other negatively.
What this means in practice
- •For statistical modelers: Better characterize uncertainty limits when modeling dependent binary events with fixed total outcomes.
- •For information theorists: Use the maximum entropy bound to optimize codes and protocols where error indicators are negatively related.
A theory result. No direct application yet.
Authors
Fraser Daly
Abstract
We show that the binomial distribution supported on $\{0,1,\ldots,n\}$ and with mean $np$ for some $0<p<1$ maximises entropy within the class of sums of negatively related indicator random variables with this mean and support. Our results generalise the main result of Yu (2008), who showed that this binomial distribution maximises entropy within the smaller class of ultra-log-concave random variables of order $n$. Our proof makes use of a Markov chain construction of Yu (2008), together with a stochastic ordering argument.