Tensor networks used to simplify complex maximum entropy distributions
Tensor network representations of discrete maximum entropy distributions via mean polytopes
Artificial IntelligenceMachine Learning
Summary
Complex probability distributions that balance uncertainty and known averages are hard to describe efficiently. The authors created a new type of tensor network called CompActNets that can represent all these distributions using a common framework. They connect the shape of possible average values, called mean polytopes, to the structure of these networks. This approach also clarifies how limiting values can reduce the range of possibilities, and shows how complexity relates to network size. They demonstrate the ideas with examples involving true-or-false logic patterns.
maximum entropy distributionstensor networksexponential familiesconvex polytopeexpectation constraintsmean polytopessupport of distributiontensor network rankBoolean statistics0/1-polytopes
Authors
Alex Goessmann, Martin Eigel
Abstract
We present tensor network representations for discrete maximum entropy distributions under expectation constraints. To this end, we introduce Computation-Activation Networks (CompActNets), a tensor network architecture that subsumes exponential families. By leveraging the geometry of the convex polytope of realizable expectation vectors, we represent any maximum entropy distribution in the same architecture. We exploit the fact that proper faces of this polytope correspond to the boundary closure of exponential families, which restricts the distribution's support. We then derive explicit representations for the support within the CompActNet architecture. The proposed framework suggests tensor network ranks as complexity measures for faces. Finally, a case study on Boolean statistics links the geometry of 0/1-polytopes directly to propositional formulas.