Complementary label aware data valuation improves noisy data detection

CAVA: Complementary-Label-Aware Data Valuation

Machine LearningComputer Vision and Pattern Recognition

Summary

Correct training requires good labels, but sometimes we only know labels a sample does not belong to, which is cheaper to get but can be wrong. The authors found that current ways to spot bad labels fail when applied directly to this type of data. They created CAVA, a method that better understands the meaning of negative labels and successfully detects errors. This helps improve training even when few clean examples are available.

What this means in practice

Authors

Tan-Ha Mai, Hsuan-Tien Lin, Vu Nguyen

Abstract

Complementary-label learning (CLL) trains a classifier from labels that name a class an instance does not belong to. Such labels are cheap to collect from crowdworkers or vision-language models (VLMs), making them attractive for large-scale annotation. However, both sources can mistakenly assign the true class as the complementary label. Such annotation errors are particularly harmful in CLL because the model is explicitly trained to suppress the correct answer. Optimal transport (OT) data valuation identifies harmful samples by comparing training data with a clean validation set, but transferring it directly to CLL creates a semantic mismatch: label $c$ means not class $c$ in the training but class $c$ in the validation. We show that this makes the OT label groups noisy mixtures whose behavior depends on the noise rate. The transferred score is informative only at sufficiently high noise, becomes uninformative at the crossover rate $1/M$ for $M$ classes, and reverses below it, where practical CLL annotation typically operates. We propose CAVA, which introduces a complementary label cost and a label margin that isolates the contribution of each complementary label to the transport potential. The margin keeps a consistent direction across noise rates, detects diagonal noise with a parameter-free threshold of closed-form precision, and allows detected samples to be recycled as ordinary-label supervision. On synthetic and human-annotated benchmarks, CAVA separates diagonal noise almost perfectly where directly transferred OT valuation performs worse than random. With only a fraction of the clean validation data, it also improves every CLL objective we evaluate, showing that OT-based valuation becomes reliable once the semantics of complementary supervision are modeled explicitly.