Tensor networks capture all discrete max entropy distributions efficiently
Tensor network representations of discrete maximum entropy distributions via mean polytopes
Artificial IntelligenceMachine Learning
Summary
Maximum entropy distributions are a way to describe the most unbiased probability patterns when you know certain average values. The authors introduce a new method called Computation-Activation Networks (CompActNets), which use tensor networks to represent these distributions in a unified way. This method relies on a geometric shape called a polytope that gathers all possible average values you can get and shows how different parts correspond to different supports of the distribution. Their approach also measures complexity by aspects of the tensor networks and links to propositional logic in special cases.
What this means in practice
- •For machine learning engineers: Represent complex discrete probability models efficiently for inference and learning using tensor network architectures.
- •For statistical data scientists: Model constrained maximum entropy distributions with clear geometric and computational tools for improved understanding and manipulation.
Authors
Alex Goessmann, Martin Eigel
Abstract
We present tensor network representations for discrete maximum entropy distributions under expectation constraints. To this end, we introduce Computation-Activation Networks (CompActNets), a tensor network architecture that subsumes exponential families. By leveraging the geometry of the convex polytope of realizable expectation vectors, we represent any maximum entropy distribution in the same architecture. We exploit the fact that proper faces of this polytope correspond to the boundary closure of exponential families, which restricts the distribution's support. We then derive explicit representations for the support within the CompActNet architecture. The proposed framework suggests tensor network ranks as complexity measures for faces. Finally, a case study on Boolean statistics links the geometry of 0/1-polytopes directly to propositional formulas.