Probability distribution evolution explained through least action principle

Physics of Information Geometry - Part I: Principle of Least Action on the Probability Simplex

Information Theory

Summary

The paper looks at how a probability distribution changes from a balanced state to an unbalanced one using a physics-inspired framework. It treats moving through possible probability states like a journey where distance measures how far from balance you are and movement costs information energy. By breaking big changes into smaller steps chosen carefully, less energy is used overall. The authors also connect this to thermodynamics, showing how the shape of the underlying probability landscape affects the costs to move through it.

What this means in practice

  • For statistical modelers: Calculate efficient transition paths between probability states under constraints, reducing computational cost of large distribution shifts.
  • For machine learning engineers: Design algorithms that update probabilistic models with minimal information cost using geometric insights on distribution evolution.

A theory result. No direct application yet.

Authors

C. Emre Koksal, Deniz Sargun

Abstract

We develop a least-action framework for describing how a probability distribution can evolve from an equilibrium state to a prescribed nonequilibrium state under constrained incremental changes. Taking a Gibbs distribution as the equilibrium reference, the framework gives a direct physical meaning to the geometry of the probability simplex: distance from equilibrium corresponds to nonequilibrium free energy, while changes between successive distributions carry an informational kinetic cost. The Pythagorean structure of relative entropy then provides the central insight of the work. It shows that intermediate distributions chosen via sequential information projections can reduce the kinetic cost of large transitions and establishes an energy-conservation-like relation between the kinetic expenditure along a path and the free energy accumulated in reaching the target distribution. Motivated by this geometry, we construct a greedy least-action path through successive information projections, obtain a closed-form characterization of each projection through the Lambert W function, and establish a finite-step performance guarantee. We further show that state-dependent costs can be incorporated naturally by reshaping the underlying Gibbs reference, providing a thermodynamic interpretation of path penalties as modifications of the effective energy landscape. Together, these results provide a unified view of distributional evolution through least action, information geometry, and nonequilibrium thermodynamics.