DisCo algebra reveals limits of summarizing joint probabilities
DisCo Algebra: Why Disintegration and Combination Are Not Inverses
Machine Learning
Summary
This paper studies how to break down and rebuild joint probability information using simple summaries. The authors show that compressing data with these summaries loses some details about how parts depend on each other. Rebuilding from the summaries can perfectly restore original data only when done in one direction, but not the other. This finding highlights a fundamental limit on summarizing complex dependencies with simple numeric summaries.
What this means in practice
- •For statistical modelers: Identify fundamental limits when summarizing joint distributions using scalar summaries to improve modeling strategies.
- •For data compression engineers: Design compression schemes that recognize and accommodate irrecoverable information loss in block-wise probabilistic summaries.
A theory result. No direct application yet.
Authors
Xuexing Lu
Abstract
We present DisCo algebra---a compositional framework for joint probability built from a symmetric cooperad (disintegration) and a symmetric operad (combination) on the bare family $\mathcal{P}_n=\mathrm{Prob}([0,1]^n)$. No probability integral transform, no external summary functions, no enrichment of objects with conditional kernels. The disintegration side uses block-wise joint CDFs as intrinsic, measure-determined 1D summaries, yielding a lossy compression. The combination side reads the same summaries and reconstructs via regular conditional measures, yielding a lossless embedding. We prove that combination followed by disintegration is the identity on admissible inputs, but disintegration followed by combination is not---the information loss is exactly the conditional KL divergence between the true conditional distribution and the conditionally independent product. This asymmetry reveals a hard limit on how much cross-block dependence can be captured by scalar block summaries.