Limitations found for splitting information when some sources are known
Partial Information Decomposition When Some Sources Are Known: A Source Chain Rule and a No-Go Theorem
Information Theory
Summary
This paper studies how to break down information from several sources about a target into parts, especially when some sources are already known. The authors examine two natural ways to do this and discover they only agree under a special rule about redundant information. They prove that when there are three or more sources, this special rule cannot hold together with the idea that all information parts should be positive. This means certain intuitive ways to split information cannot work perfectly in all cases.
What this means in practice
- •For information theorists: Clarify limits on how to break down shared information among multiple sources, guiding theoretical measures of redundancy.
- •For data fusion engineers: Provide theoretical constraints on combining data from multiple sensors, affecting strategies for extracting shared versus unique information.
A theory result. No direct application yet.
Authors
Aaron J. Gutknecht
Abstract
Partial information decomposition (PID) splits the information that several sources carry about a target into atoms, distinguished by which collections of sources give access to them. We ask how such a decomposition should behave when some of the sources are already known. The access structure of the atoms suggests one answer: discard the atoms that the known sources provide and regroup the rest according to which further sources unlock them. The classical construction of conditional information suggests another: recompute the decomposition under the conditional distributions and average. We show that the two answers agree, atom by atom, exactly when redundant information obeys a source chain rule, an analogue of the chain rule of mutual information: when the same sources are added to several collections, the enlarged collections share everything the added sources provide and, beyond that, exactly what the original collections share once the added sources are known. The rule divides established measures of redundant information, so that adopting or rejecting it genuinely narrows the choice of measure. Our main result is a no-go theorem: for three or more sources, the source chain rule and local positivity, the requirement that all atoms be nonnegative, cannot hold together in any PID.