Long-tailed recognition improved by keeping class diversity in features
Full Collapse Is Suboptimal: Controlled Partial Neural Collapse for Long-Tailed Recognition
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
When teaching computers to recognize many categories, some rare categories get less attention and their features can become too similar, hurting performance. The authors show that forcing all categories to have very similar features is not ideal for rare classes. They propose a new way that allows common classes to have tightly grouped features while rare classes keep some variety. This approach improves recognition accuracy on imbalanced image datasets.
What this means in practice
- •For computer vision engineers: Build image classification models that better handle rare categories by balancing feature compactness and diversity.
- •For machine learning platform teams: Incorporate adaptive margin loss and class-aware feature specialization to improve performance on long-tailed datasets.
Authors
Hyeontaek Hwang, Cheonghwan Hur, Daeyoung Kim
Abstract
Recent long-tailed recognition (LTR) methods incorporate neural collapse (NC) and align features to a fixed Equiangular Tight Frame (ETF) as a uniform target for all classes. However, forcing tail classes to fully collapse over-contracts their samples and weakens generalization. To address this, we propose a controlled partial neural collapse, realized through two coordinated controls on a backbone: the Class-Adaptive Sparse Activation (CASA) classifier and the ETF-constrained adaptive margin (ETF-AM) loss. Rather than forcing a uniform full collapse, this regime allows head classes to collapse along the ETF while tail classes retain within-class diversity. This prevents the tail from being forced into full collapse, which is suboptimal under imbalance. To set the dimensionality of collapse, CASA induces class-related node specialization through feature-conditioned gating so that tail classes occupy their own feature directions. For the clearance axis, ETF-AM applies a frequency-adaptive margin, with smaller margins for head classes and larger for tail classes. On CIFAR100-LT, ImageNet-LT, Places-LT, and iNaturalist-2018, our method achieves the best overall accuracy over prior methods. In the empirical analysis, within-class spread increases monotonically from head to tail classes, consistent with tail classes retaining variability rather than fully collapsing.