Knowledge fusion network improves continual learning without forgetting
Learning without Overwriting: Structural Knowledge Fusion for Continual Learning
Machine LearningArtificial Intelligence
Summary
When teaching computers new tasks one after another, they often forget what they learned before. The authors introduce a Knowledge Fusion Network that adds small new parts for each task instead of changing everything in one big model. This approach helps the computer remember earlier tasks better and handle new ones without confusion. It also uses fewer resources compared to retraining from scratch or doubling the model size.
What this means in practice
- •For machine learning engineers: Develop systems that learn new image recognition tasks sequentially without losing accuracy on previous tasks.
- •For satellite data analysts: Improve classification accuracy on evolving satellite image datasets by adding new task knowledge without compromising old data models.
Authors
Kanishk Bakshi, Amitansu Pattanaik, Kathiravan Srinivasan
Abstract
Catastrophic forgetting in sequential deep learning arises from a fixed-capacity architectural constraint: parameter updates encoding new tasks perturb weight subspaces supporting prior representations. We introduce the Knowledge Fusion Network (KFN), which mitigates the stability-plasticity trade-off in class-incremental learning through novelty-gated structural expansion rather than parameter negotiation within a shared capacity budget. KFN decouples task acquisition from knowledge consolidation: each incoming task is learned in an isolated lightweight specialist network (2.8M parameters) and integrated into the global model via a Structural Fusion Module that allocates dedicated channel capacity conditioned on a quantitative novelty score. Dual structural isolation--channel-space separation with temporally phased gradient unfreezing--protects consolidated knowledge. On ten-task Split CIFAR-100, KFN achieves 11.94% minimum task accuracy and 71.16% task-10 accuracy, whereas shared-parameter baselines degrade severely: LwF to 9.07% average accuracy (0.00% min), Replay++ to 8.91% (85.08% forgetting), and Replay to 16.21% (73.44% forgetting); KFN's forgetting (57.92%) is 15.5 pp lower than Replay. Furthermore, KFN prevents logit-scale drift, maintaining near-symmetric task performance. Across CIFAR-100 and EuroSAT satellite imagery, KFN's class-incremental retention (72.7% and 42.5%) substantially exceeds the best baselines (29.6% and 0.94%). Crucially, KFN achieves parameter-efficient linear scaling, adding 25% of the base parameter count per step and requiring 25.1% of naive backbone duplication over ten tasks (28.1M vs 112.0M). With a 37.1% peak GPU memory reduction over full retraining, KFN establishes novelty-gated expansion as a practical path beyond fixed-capacity architectures.