Probability distributions change with least action and energy costs
Physics of Information Geometry - Part I: Principle of Least Action on the Probability Simplex
Information Theory
Summary
This paper looks at how probability distributions, which show how likely different outcomes are, change from a calm balanced state to a more active one. The authors use ideas from physics, like energy and motion, to understand these changes as happening over the simplest or 'least costly' paths. They link geometric ideas about distance in probability space to physical ideas like free energy and informational effort, showing that splitting big changes into smaller steps can save energy. The work connects concepts from information theory, geometry, and thermodynamics in a new unified way.
Probability simplexGibbs distributionNonequilibrium thermodynamicsRelative entropyInformation geometryLeast action principleFree energyInformation projectionLambert W functionKinetic cost
Authors
C. Emre Koksal, Deniz Sargun
Abstract
We develop a least-action framework for describing how a probability distribution can evolve from an equilibrium state to a prescribed nonequilibrium state under constrained incremental changes. Taking a Gibbs distribution as the equilibrium reference, the framework gives a direct physical meaning to the geometry of the probability simplex: distance from equilibrium corresponds to nonequilibrium free energy, while changes between successive distributions carry an informational kinetic cost. The Pythagorean structure of relative entropy then provides the central insight of the work. It shows that intermediate distributions chosen via sequential information projections can reduce the kinetic cost of large transitions and establishes an energy-conservation-like relation between the kinetic expenditure along a path and the free energy accumulated in reaching the target distribution. Motivated by this geometry, we construct a greedy least-action path through successive information projections, obtain a closed-form characterization of each projection through the Lambert W function, and establish a finite-step performance guarantee. We further show that state-dependent costs can be incorporated naturally by reshaping the underlying Gibbs reference, providing a thermodynamic interpretation of path penalties as modifications of the effective energy landscape. Together, these results provide a unified view of distributional evolution through least action, information geometry, and nonequilibrium thermodynamics.