Graph neural networks learn new classes without forgetting old ones
MAEM: Mitigating Recency Bias via a Replay-Free, Capacity-Controlled Dynamic Mixture of Experts for Class-Incremental Graph Learning
Machine Learning
Summary
Graph neural networks struggle to remember old knowledge when learning new classes one after another. The authors introduce MAEM, a method that assigns each group of new classes to a separate expert model that is then frozen to protect past knowledge. Instead of storing old data, MAEM uses a smart way to pick the best expert for new inputs and merges less-used experts when needed to keep the model size manageable. This approach helps reduce the problem of recent classes overshadowing older ones, improving performance on graphs without needing to review old data.
What this means in practice
- •For machine learning engineers: Implement continual graph learning systems that learn new classes without revisiting previous data or requiring task identifiers.
- •For network operations teams: Maintain accurate graph-based network models that adapt to new device types or behaviors incrementally while preserving earlier learned patterns.
Authors
Junlin Chen, Wenhao Wang, Junchao Xu, Qingsong Yao, Fei Xue, Li Sun, Xiao Wang, Ruijie Wang
Abstract
Graph Neural Networks (GNNs) excel on graph-structured data but suffer catastrophic forgetting when classes arrive sequentially. In class-incremental graph learning, predictions are often dominated by recently learned classes, causing severe recency bias. Dynamic expansion can isolate knowledge across sessions, but many methods assume task-incremental settings and require task identities at test time to select branches or heads---information unavailable in class-incremental learning. Learnable routers offer an alternative, yet continual fine-tuning the router on new data can bias routing toward recent components. We propose MAEM, a replay-free, capacity-controlled dynamic mixture-of-experts framework for class-incremental graph learning. MAEM builds representations with Simplified Graph Convolution (SGC) and trains a dedicated expert for each new session, then freezes it to protect prior knowledge. For routing without task IDs or a shared trainable router, each expert uses a masked autoencoder (MAE); test nodes are assigned to the expert with the lowest reconstruction error, reducing routing-level recency bias. When the expert pool reaches a fixed capacity, MAEM merges the least-used experts via Git Re-Basin alignment and further distills them into a fused expert using topology-aware pseudo data synthesized from MAE distributions (DeepInversion-style), without storing or revisiting historical graph data. On four class-incremental graph benchmarks, MAEM mitigates recency bias, outperforms strong continual graph learning baselines, and keeps model capacity bounded under strict replay-free learning.