Rethinking Transfer in Continual Learning: A Replay-Based Realisation
Machine Learning
Summary
The authors study how language models can keep learning new tasks over time without forgetting old ones. They point out that most methods focus only on avoiding forgetting, but don't really explain when learning from past tasks should help new ones. They propose three conditions that need to be met for transfer learning to work well and create a method called Transfer-Selective Replay (TSR) that selectively reuses helpful past data. Their approach improves learning new tasks while keeping old knowledge intact, doing better than existing methods in different task setups. Overall, the authors suggest making transfer learning an important goal in continual learning research.
Authors
Yang Meng, Zhenya Liu, Zhuokai Zhao, Yuxin Chen
Abstract
Continual learning studies how deployed language models can continually acquire new tasks without expensive retraining from scratch. Existing methods, whether rehearsal-based (replaying stored past data) or rehearsal-free (regularising or isolating parameters), overwhelmingly target one objective: preventing catastrophic forgetting. Forward transfer, the past helping the future, has meanwhile been pursued almost exclusively through parameter reuse, with no explicit account of when transfer should be expected at all. We begin one step earlier: before designing a transfer mechanism, we ask when transfer should exist at all. We answer with a framework of three measurable conditions: the target task must leave room for improvement beyond its own limited supervision, transferable information must survive continued optimisation, and replay must come from compatible previous tasks. We instantiate this view as Transfer-Selective Replay (TSR), which selects replay data predicted to benefit the incoming task rather than replaying past examples indiscriminately. Selection is guided by a zero-training task signature, while distillation preserves stability on previous tasks. Under the standard continual learning protocol in the low-budget regime, TSR consistently improves forward transfer while maintaining stability, outperforming existing replay baselines across heterogeneous and homogeneous task streams. More broadly, the results argue for treating transfer as a first-class objective of continual learning, to be understood before it is engineered.