Statistical framework models nonequilibrium probability distributions with energy and entropy constraints
Distributional statistics of nonequilibrium
Information Theory
Summary
Sometimes, scientists want to understand how systems behave when they are not in balance, like a cup of hot coffee cooling down. This paper presents a way to think about all the possible ways a system can be arranged when we know some key numbers, like average energy and disorder. The authors build mathematical tools to describe these nonequilibrium situations more generally than before. Their framework also connects smoothly with established ideas about systems at balance and can be used for various types of analysis and inference.
What this means in practice
- •For statistical physicists: Generate reference distributions representing nonequilibrium states constrained by energy and entropy for modeling physical systems away from equilibrium.
- •For machine learning engineers: Incorporate tunable nonequilibrium priors in probabilistic models to improve inference tasks where data distributions deviate from ideal equilibrium assumptions.
Authors
Harrison Hartle, Aanjaneya Kumar, David H. Wolpert
Abstract
We establish a reference modeling framework for random probability distributions subject to constraints. Generalizing the hyperensemble approach of Crooks, we formulate Jaynesian priors of maximal uncertainty over the unit simplex of categorical distributions. This framework is entirely general, but we primarily investigate models with constraints on physically relevant quantities, namely, average energy and Shannon entropy. A constraint on the exact value of average energy uniquely determines a Boltzmann-Gibbs equilibrium, but also defines a hyperplanar subset of the probability simplex consisting of all distributions with that value of average energy. The associated hyperensemble has a uniform probability density across all such distributions, regardless of distance to equilibrium. Such reference models make predictions of generic nonequilibrium properties such as the mean and modal microstate distributions and the expected free energy of a random distribution, which we examine analytically and via exact sampling algorithms. Moreover, introducing constraints on Shannon entropy permits parametric access to tunably nonequilibrium regimes. The approach follows conventional Jaynesian modeling logic and is compatible with standard Bayesian inference, and reduces to Boltzmann-Gibbs statistics when Shannon entropy is maximized. We examine the physical predictions of this scheme, and demonstrate its wide compatibility with other approaches to nonequilibrium modeling. We illustrate the framework in three scenarios: (i) comparison to superstatistics quasi-equilibrium modeling, (ii) generation of random nonequilibrium steady states, and (iii) Bayesian inference under a prior hyperensemble. We highlight numerous prospects for theoretical extensions and empirical application.