Incremental learning system balances new and old knowledge effectively
Co2In: a Bi-level Memory Incremental Learning Framework with Knowledge Encoding, Consolidation, and Integration
Artificial Intelligence
Summary
Learning new things without forgetting old ones is hard for artificial intelligence. The authors present a new method called Co2In that mimics how human memory works, using short-term and long-term memory modules. This method first learns new data quickly, then organizes and stores knowledge more efficiently for long-term use. Their approach improves learning performance on image recognition tasks while using memory efficiently.
What this means in practice
- •For machine learning engineers: Build image classification systems that update continuously without retraining on all old data, improving efficiency and accuracy.
- •For robotics software teams: Implement robotic systems that can adapt to new environments over time while retaining previously acquired skills without large memory overhead.
Authors
Wenju Sun, Qingyong Li, Boyang Li, Wen Wang, Siyu Zhang, Yi Wei, Yangliao Geng
Abstract
Incremental learning (IL) aims to continually acquire new knowledge (plasticity) while retaining previously learned information (stability). However, striking a balance between plasticity and stability remains a significant challenge for intelligent systems. The human brain achieves exceptional balance, owing to various memory units that collaboratively encode and store information. Inspired by human memory mechanisms, this paper introduces an IL framework with knowledge enCoding, Consolidation, and Integration (Co2In). Co2In is designed as a bi-level memory architecture: a working memory for adaptive knowledge acquisition and a long-term memory dedicated to the persistent retention of information. The two memory modules work cooperatively during IL. The working memory first learns from new data to encode knowledge into parameters. Subsequently, Co2In performs a consolidation process to identify underlying patterns in the learned parameters and re-express them into a compact knowledge representation. Next, the knowledge representation and the identified patterns are transformed into separate network layers and integrated into the long-term memory. These designs empower Co2In to accumulate knowledge with high plasticity and stability. We evaluate Co2In on CIFAR-10, CIFAR-100, and Tiny-ImageNet under exemplar-free Class-IL and Task-IL settings. Experimental results show that Co2In achieves state-of-the-art performance with efficient memory consumption.