Leveraging Dissimilarity Invariance as a Robust Anchor for Learning with Noisy Labels
2026-07-20 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors found that deep learning models struggle when the labels they are trained on have mistakes, especially when trying to recognize similar items. They discovered that even if labels are noisy, the differences between unrelated items stay consistent, which they call Dissimilarity Invariance. Using this idea, they created a method called NegScale that focuses on learning what is different rather than what is similar. Their approach improves learning by making unrelated groups distinct and reducing false similarities, which they proved theoretically and showed works better than previous methods on noisy image datasets.
deep learninglabel noisevisual recognitionsimilarity learningdissimilarity invarianceorthogonality penaltysubspace orthogonalityCIFAR datasetnoise robustnessrepresentation learning
Authors
Wenxiao Fan, Kan Li
Abstract
Deep learning models excel in visual recognition but suffer severe performance drops when training labels are corrupted by noise. Under label noise prior work cannot learn accurate similarities and thus misguide the learning process. In this paper, we uncover a complementary and novel phenomenon, Dissimilarity Invariance, whereby semantic dissimilarity between unrelated samples remains stable despite label noise. Leveraging this insight, we propose NegScale, a plug-and-play framework that shifts focus from fragile similarity to robust dissimilarity. NegScale integrates: (1) Structured Negative Orthogonality Penalty (SNOP), enforcing subspace orthogonality for unrelated samples; and (2) Dissimilarity-Calibrated Similarity Adjustment (DCSA), suppressing spurious similarity using dissimilarity anchors. We also give theoretical analysis that proves Dissimilarity Invariance and the effectiveness of NegScale. Empirical results demonstrate that NegScale consistently outperforms state-of-the-art baselines, establishing new benchmarks on CIFAR with synthetic noise and real-world datasets.