Server method keeps AI from forgetting old tasks in federated learning
ReSCENE: Server-Side Replay for Structural Mitigation of Catastrophic Forgetting in Federated Continual Learning
Machine Learning
Summary
When multiple devices train a shared AI model step-by-step with new tasks, the model can forget what it learned before. The authors propose ReSCENE, a way for each device to send a small summary of its data to a central server. The server then uses these summaries from past and current tasks to train the AI, helping it remember old knowledge without overloading the devices. This approach saves device power and internet data while improving overall accuracy, making continual learning more practical.
What this means in practice
- •For edge device engineers: Use server-side data summaries to reduce device processing and data upload during federated continual learning.
- •For cloud infrastructure teams: Deploy ReSCENE to maintain accuracy across multiple AI tasks while controlling server memory by compressing client data summaries over time.
Authors
Sungmin Kang, Zhengzhong Tu, Sunwoo Lee
Abstract
Federated continual learning must integrate new tasks over time without losing earlier-task knowledge. Most existing methods attach an anti-forgetting mechanism to the client-trained, server-aggregated loop of federated learning, which holds back new learning to preserve earlier knowledge and burdens resource-constrained clients. We propose ReSCENE, which structurally mitigates catastrophic forgetting by having each client upload a small condensed surrogate of its local data while the server keeps the surrogates of past tasks and trains the global model on them together with the current task surrogates. For efficient server memory, we introduce temporal herding, which selects the more recent surrogates from the pool accumulated over a task into a compressed buffer. Our study provides a theoretical analysis showing that this buffer can represent the original task data more closely than full accumulation of all surrogates. Across CIFAR-10, CIFAR-100, and TinyImageNet, ReSCENE achieves the strongest accuracy over seven baselines, by up to $31.1$ points of average accuracy, while requiring as little as $0.11\times$ of the client computation and up to $179\times$ less upload than the model-update baselines. ReSCENE further demonstrates its effectiveness when scaled to larger client populations and larger models while remaining efficient, which makes it a practical method for federated continual learning.