Freezing feature extractor reduces forgetting in sound classification models
Investigating catastrophic forgetting in sound event classification
Artificial IntelligenceSound
Summary
When teaching computers to recognize new sounds one after another, the problem is they often forget what they learned before, which is called catastrophic forgetting. The authors looked at ways to stop this forgetting during sound recognition tasks. They found that freezing the parts of the model that identify general features while only adjusting the last part that decides the sound type helps prevent forgetting and keeps the model stable. This solution works well in their tests and balances keeping old knowledge with learning new sounds.
What this means in practice
- •For audio software developers: Maintain accuracy in sound recognition apps when updating models with new sound classes incrementally.
- •For machine learning engineers: Improve stability in continuous learning systems for audio event classification by freezing feature extraction layers.
Authors
Riccardo Casciotti, Annamaria Mesaros
Abstract
This work investigates a number of approaches to prevent catastrophic forgetting in class incremental learning scenarios for sound event classification tasks. We analyze the problem using architectural and regularization approaches, using FSD50K and AudioSet datasets. We design incremental stages and solutions that selectively protect the kernels of the network from weight updates to prevent catastrophic forgetting, and a dynamic head solution that expands itself each time a new task is learned. The findings show that catastrophic forgetting mainly happens in deeper layers, in particular in the classifier head. For the studied in-domain sound classification problem, the solution that seems to alleviate catastrophic forgetting and is the most efficient is a full freezing of the feature extractor with a fine-tuning of the dynamic head classifier, showing little to no forgetting and great training stability, and a good balance between memory-stability and learning plasticity.