Bregman geometry reveals new views on exponential family distributions

Bregman geometry in regular exponential families: dual representations and relative smoothness

Machine Learning

Summary

Exponential family distributions are common tools in statistics with neat mathematical features. The authors study these distributions through the lens of Bregman geometry, which connects geometry with statistics and information theory. They develop new geometric representations that clarify how these distributions behave when described by their average values. These ideas also help understand related statistical models and improve methods for working with them, like in the example of the Poisson distribution.

What this means in practice

A theory result. No direct application yet.

Authors

Mame Diarra Fall, Éric Barat, Nicolas Dobigeon

Abstract

Regular exponential-family distributions are ubiquitous in statistics due to their appealing and convenient properties. Notably, they can be expressed in terms of Bregman divergences, revealing fundamental connections between statistics, convex analysis, and information geometry. However, this geometric structure has not been systematically exploited to characterize these distributions with respect to their expectation parametrization. Building on a projective characterization, we establish a general construction of equivalent dual Bregman representations, together with normalization and uniqueness results. These representations express log-densities as ordinary Bregman divergences in dual coordinate systems, providing geometric characterizations of the parameter space. So-called $β$- and $α$-scaled dual representations emerge as particular instances of this framework. We further derive dual Bregman representations of conjugate priors and establish that they preserve conjugacy. Applications to classical distributions illustrate the geometric structures underlying seemingly different statistical models. Moreover, we show that these representations provide natural reference geometries for establishing relative smoothness of the primal Bregman representation of the distribution, with explicit bounds that are sharp in the scalar and separable settings. An illustrative application to the Poisson distribution sheds new light on several works in the literature, providing complementary geometric justifications for reference geometries while deriving existing algorithms.