Optimal equation helps balance learning stability and flexibility
A Bellman Optimality Equation for Plasticity
Machine Learning
Summary
Balancing learning stability and the ability to adapt, called plasticity, is a key challenge in AI systems that learn continually. The authors build on previous work that defined plasticity and empowerment in terms of information flow between an AI agent’s actions and observations. They show there is a mathematical equation, like the famous Bellman optimality equation, that can help optimize plasticity in decision-making processes. This is a first step toward better managing how much AI agents change their behavior over time.
What this means in practice
- •For robotics engineers: Design adaptive control systems that better balance continuous learning stability with flexibility in changing environments using the plasticity optimization equation.
- •For game ai developers: Improve non-player character behaviors by optimizing how they adjust actions over time to achieve more realistic and stable learning.
A theory result. No direct application yet.
Authors
Jeremy Lucas, Doina Precup
Abstract
In continual reinforcement learning, carefully managing the stability-plasticity tradeoff remains a core challenge. Recent work by Abel et al. (2025) formalized this dilemma by defining plasticity as the generalized directed information from an agent's observations to its actions, and empowerment as the generalized directed information from its actions to its observations. This formulation successfully reframes the traditional stability-plasticity tradeoff as an empowerment-plasticity tradeoff. However, while extensive literature exists on optimizing for empowerment, there is currently no research addressing the optimization of plasticity under this new definition. This paper presents preliminary work toward optimizing plasticity within Markov decision processes. We show that there exists a Bellman optimality equation for optimizing plasticity similar to previous work for empowerment.