Dynamic width expansion helps neural networks learn new classes better
Width Expansion as a Method for Class Incremental Learning
Machine LearningArtificial Intelligence
Summary
When computers learn new categories of things over time, they often forget what they learned before. The authors propose a way for neural networks to add more neurons only when they need to, which helps them remember old information while learning new stuff. They also use a special memory-attention system to keep the new and old knowledge from mixing up too much. This approach was tested on image tasks and worked better than keeping the network size fixed.
What this means in practice
- •For deep learning engineers: Build neural networks that adaptively grow with new data classes without needing task labels, improving continual learning performance.
- •For ai system integrators: Develop AI that updates its knowledge over time in changing environments without storing old data or needing to know when tasks change.
Authors
A. L. S. Conde, Y. Elkhatib, C. M. Ranieri
Abstract
Class Incremental Learning (Class-IL) requires models to learn new classes over time while preserving previously acquired knowledge without access to past data or task identity. This setting intensifies the stability-plasticity dilemma and makes catastrophic forgetting a central challenge. Existing approaches include regularization, knowledge distillation, replay, and architectural expansion. However, many expansion methods rely on explicit task identifiers or predefined growth strategies, limiting their applicability when task boundaries are unavailable at inference time. This work proposes a dynamic width expansion method that increases the number of neurons within existing layers according to a normalized loss criterion, without requiring task-specific information. An attention mechanism with persistent key-value memory is also incorporated to stabilize feature representations and reduce interference between previously learned and newly introduced classes. The approach is evaluated on Split MNIST and Split CIFAR-100 under the standard Class-IL protocol. Experiments compare fixed-capacity and dynamically expanding architectures, both with and without attention, combined with established continual learning methods including EWC, LwF, and A-GEM. Results show that progressive width expansion consistently improves performance over fixed architectures, particularly when combined with functional methods and A-GEM. The combination of width expansion and attention provides the most consistent gains. Overall, dynamic width expansion based on representational demand provides an effective and flexible strategy for Class-IL, although uncontrolled growth may increase overfitting and computational cost.