Evolving specialized memories helps reinforcement learning handle changing tasks
Mitigating Catastrophic Forgetting with Physics-Indexed Specialist Memory: the Evolutionary Neuroplastic Agent
Machine LearningArtificial IntelligenceNeural and Evolutionary Computing
Summary
Machines that learn from experience struggle to keep skills when the rules of the game change. The authors created a learning agent that stores small expert skills tied to different situations and picks the best one when needed. They tested it in changing game-like tasks and found it kept older skills more consistently than some simpler methods. However, it was slower and sometimes less effective than other advanced learning methods. The work shows a way to remember past solutions but doesn't prove it is the best for all cases.
What this means in practice
- •For robotics engineers: Maintain performance of control systems in robots that face varying environments by switching between evolved specialist policies.
- •For game ai developers: Use archived expert behaviors to keep AI agents effective when game dynamics or rules change over time.
Authors
Blas Eugenio Vicco Zarate, Miguel Hernán Vicco
Abstract
Continual reinforcement learning requires adaptation to changing conditions while preserving performance under previously encountered dynamics. It was investigated whether specialists generated by evolutionary search can provide a reusable memory when combined with environment detection and failure-driven search. The Evolutionary Neuroplastic Agent (ENA) evolves small neural networks without gradients, archives selected policies by detected dynamic regime, and retrieves them through Bayesian policy reuse. The architecture was evaluated in non-stationary CartPole and HalfCheetah environments. In the 30-replicate CartPole ablation, the complete architecture achieved a mean success rate of 83.2% across the five test environments. Among variants that learned the harder CartPole environments, the complete architecture showed the most consistent retention. A selection ablation also showed that the retrieval rule affected performance. In the evaluated sequences, ENA adapted more slowly than PPO on CartPole and achieved lower returns than SAC on HalfCheetah. The evidence concerns the complete architecture and does not isolate gradient-free optimization as the cause of retention or establish superiority over other continual-learning methods.