Audio feature selection improves few-shot learning in shifted settings

Sample-Conditioned Representation Selection for Audio Few-Shot Learning

Artificial IntelligenceSound

Summary

Few-shot audio classifiers sometimes get confused when sounds in the background change or mix with the target sound in new ways. The authors found that for a common audio model, only a small number of features explain most of these confusing shifts. They created a method called SAMPLESELECT that picks important features for each sound example without retraining the whole model. This method improves how well the model recognizes new sounds, especially when backgrounds change, by focusing only on the useful parts of the audio representation.

What this means in practice

Authors

Fengrui Liu, Ningxin Shen, Yi Li, Yiwei Fu, Feng Liu, Jiangmeng Li

Abstract

Few-shot audio classifiers may rely on foreground-background co-occurrences and fail when those correlations shift. On SpurAudio, the resulting representation shift is concentrated and class dependent: for ResNet12, the top 10 percent of channels explain 82.80 percent of the null-corrected shift contribution. We propose SAMPLESELECT, which predicts a fixed-budget feature mask independently for each input while keeping the encoder and source classifier frozen. Training uses differentiable Gumbel Top-k selection with foreground classification and cross-background contrastive losses; inference uses deterministic Top-k masks and support-only linear adaptation. Across ResNet12 and Conv64 in 5-way 1-shot and 5-shot evaluation, SAMPLESELECT gives the best OOD accuracy among the compared methods and improves the matched full-representation control by 4.90-8.38 percentage points. Ablations and representation analyses further support the learned selection mechanism. Code is available at https://github.com/Cross-Innovation-Lab/SAMPLESELECT/
View PDFOpen arXiv