Small-step belief updates optimize distribution changes on probability simplex
Physics of Information Geometry - Part II: Small-Step Active Inference on the Probability Simplex
Information Theory
Summary
This paper studies how an agent can update its beliefs step-by-step when changing from one probability distribution to another. The authors treat these updates like movements on a geometric surface shaped by probabilities. They show that making many small, careful updates is better than jumping directly to the target belief. The work uses ideas similar to physics, like energies and paths of least resistance, to describe these belief changes. This method results in a clear formula for making the best small update toward a goal belief.
What this means in practice
- •For machine learning engineers: Design inference algorithms that update probability models in small, optimal steps to efficiently reach preferred beliefs.
- •For robotics control teams: Implement belief updating methods that move state estimates gradually and optimally for better robot decision-making under uncertainty.
A theory result. No direct application yet.
Authors
C. Emre Koksal, Deniz Sargun
Abstract
This paper is the second in a two-part investigation of the physics of information geometry. While Part I develops a physical foundation for distributional motion on the probability simplex, the present paper studies how that framework manifests in active inference. The treatment is fully self-contained and does not require familiarity with Part I. We focus in particular on active inference through small distributional steps and the geometric structure induced by such local motion. Starting from an initial distribution, an agent evolves its belief state toward a final target distribution through a sequence of constrained updates. We define a relative free energy functional with respect to the preferred distribution and extend it to a relative potential energy analogous to the Helmholtz/Gibbs free-energy decomposition. The evolution is subject to a per-step kinetic constraint expressed through the Kullback-Leibler (KL) divergence between consecutive distributions, which serves as a discrete kinetic energy on the probability simplex. Using the information-geometric Pythagorean theorem on KL balls, we show that sufficiently small local moves dominate large direct jumps, and that greedy maximization of free-energy reduction is globally optimal under the kinetic constraint. This leads to a sequential variational principle in which the optimal trajectory minimizes the associated Lagrangian of the optimization problem. Similar to classical mechanics, the Lagrangian takes on the form as the difference between the kinetic and potential terms, establishing a least-action principle for distributional motion on the simplex. The resulting optimal update admits a closed form as an exponentially tilted version of the current distribution toward the preferred distribution, parametrized by an inverse-temperature-like multiplier. We further extend the framework to incorporate state-dependent geodesic...