New entropy inequality tightens bounds for log-concave distributions
Sharp Rogers--Shephard Inequalities for Rényi Entropy
Information Theory
Summary
This paper finds a precise formula that describes how spreading out or mixing certain types of data behaves from an information standpoint. The authors focus on a mathematical property called Rényi entropy, which generalizes how we measure the unpredictability or randomness of data. They confirm a conjecture related to Shannon entropy and identify exactly when equality holds, helping to understand such spread in higher dimensions. Their proof uses geometric ideas and reveals how certain noise types affect information flow.
What this means in practice
- •For data compression engineers: Improve theoretical bounds on encoding schemes by understanding entropy inequalities for log-concave sources with noise.
- •For information theorists: Use sharp entropy inequalities to better characterize limits of communication channels with asymmetric noise.
A theory result. No direct application yet.
Authors
Bang-Xian Han, Guoqing Zhan, Chiyu Zhou, Zhuo-Nan Zhu
Abstract
We establish sharp weighted Rogers--Shephard inequalities for Rényi differential entropy. For independent copies $X,X'$ of a log-concave random vector in $\R^n$, we prove $h_p(aX-bX')\le h_p(X)+n\log(a+b)$ for all $a,b>0$ and $1\le p\le\infty$, where orders $1$ and $\infty$ correspond to Shannon entropy and min-entropy, respectively. The Shannon case settles the entropic Rogers--Shephard conjecture of Madiman and Kontoyiannis and its higher-dimensional formulation. For $1\le p<\infty$, equality holds precisely for invertible affine images of random vectors with independent exponential coordinates. At $p=\infty$, the larger equality class is characterized by possibly extended-valued convex potentials that are positively homogeneous of degree one. For every $0<p<1$, uniform laws on sufficiently high-dimensional simplices violate the unweighted bound. Thus $p=1$ is the critical order for this bound to hold in every dimension. The proof combines weighted convex intersections with an exponential lift to a log-affine probability density. At the Shannon endpoint, this lift yields an exact deficit decomposition with a conditional mutual information term for an additive channel with asymmetric Laplace noise.