Spatial coherence improves reliable memory replay in continual learning
Reliable Replay through Spatial Coherence in Online Continual Learning
Machine Learning
Summary
When computer models learn new things over time, they often forget what they learned before. This paper shows a new way to decide which memories the model should replay, by looking at how groups of memories change together instead of just focusing on single memories. The researchers created a method called SPHERE that spreads attention across related memories and keeps the model from forgetting earlier tasks. They tested this approach on tasks like recognizing images with noise, tuning language models, and code generation, and it helped models remember better and be more accurate.
What this means in practice
- •For machine learning engineers: Improve model training workflows by allocating memory replay more effectively to reduce forgetting during sequential task learning.
- •For ai model developers: Enhance continual language model tuning and reinforcement learning by using spatial coherence to optimize replay memory management.
Authors
Haixiang Sun, Jiefu Zhang, Yinghao He, Yang Xu, Vaneet Aggarwal, Bharat Bhargava, Andrew L. Liu
Abstract
Continually adapting models to new tasks requires retaining earlier knowledge under limited memory and computation. Experience replay addresses this challenge, but priorities based on individual loss increases overlook how related memories respond to the same update and can overemphasize isolated responses. We introduce SPatial coHErent risk control for REplay (SPHERE), a general replay-allocation method applicable across a broad range of learning settings. SPHERE uses a representation kernel to aggregate signed prospective loss changes, attenuating unsupported spikes while retaining coherent increases. It then formulates allocation as entropy-regularized transport, redistributing uniform source mass toward supported high-risk regions while penalizing long-distance transfers. We derive replay coefficients from the transport objective's sensitivity to the original loss changes and blend them with uniform replay to maintain baseline rehearsal. Our analysis establishes conditions under which kernel aggregation improves risk estimation and bounds transport-value inflation due to residual noise and smoothing bias. Experiments demonstrate that SPHERE improves accuracy and reduces forgetting across noisy-label vision tasks, continual language-model instruction tuning, and code-generation reinforcement learning with incomplete test rewards.