Matrix measures improve understanding of information dependence limits
Matrix $Φ$-Entropic Measures of Dependence
Information Theory
Summary
Understanding how information depends on each other helps improve data processing and communication systems. This paper extends existing scalar measures of dependence to matrices, allowing a more detailed view. The authors show how these new matrix measures behave under certain information transformations and how they can be used to find exact limits in communication tasks. They also provide examples where matrix-based measures give stronger results than traditional scalar ones.
What this means in practice
- •For data transmission engineers: Determine precise limits on information loss over communication channels using advanced matrix dependence measures.
- •For machine learning model developers: Use matrix-based dependence metrics to better understand and optimize feature interdependencies in complex data.
A theory result. No direct application yet.
Authors
Chenyu Wang, Amin Gohari
Abstract
We introduce matrix $Φ$-ribbons, extending the scalar $Φ$-ribbons to the matrix case. For $Φ$ in the subadditivity class, we establish data processing and tensorization properties of matrix $Φ$-ribbons. We establish scalar and matrix $Φ$-entropy inequalities and use them to show that both $Φ$-ribbons characterize the corresponding $Φ$-strong data processing inequality (SDPI) constants. We obtain exact evaluations of the SDPI constants for several sources. We provide examples showing that the matrix $Φ$-SDPI constant can exceed the scalar counterpart. Finally, we derive inner bounds for $Φ$-ribbons from prescribed independence structures and prove that these bounds are optimal when $Φ(t)=t\log t$.