Anomaly ranking slows learning of new categories in AI classifiers
Anomaly-ranked confirmation slows new-class learning
Machine Learning
Summary
When computers learn to recognize new types of things, people often check the examples the computer finds most unusual first. This study shows that focusing on confirming these unusual examples actually slows down how quickly the computer learns about the new type. The researchers found that picking examples more randomly or after the computer starts learning helps improve how well it recognizes the new class. They also showed that if the new type is very different from known types, fewer examples need to be checked. So, learning what a new thing is about is not the same as just finding unusual examples.
What this means in practice
- •For medical data teams: Improve selection of patient cases for expert review when training diagnostic classifiers by not solely relying on highest anomaly scores.
- •For computer vision engineers: Design better training workflows for image classifiers that recognize new object types by balancing example confirmation beyond the top anomaly rankings.
Authors
Argyro Sasli, Michael Coughlin
Abstract
Anomaly detectors rank unfamiliar examples for expert confirmation, and the confirmed examples teach a classifier to recognise a new class. Across five benchmarks from physics, medicine and computer vision, the members flagged by the detector are those the classifier learns first. In three systems, confirming the highest-ranked members of a new class gives lower recognition than confirming the same number of random members. Handing subsequent selection to the promoted classifier recovers part of this loss in two systems. A paired intervention further shows that replacing nearby known classes with distant ones sharply reduces the confirmations needed for recognition. These results distinguish finding a new class from selecting examples that teach its extent: within the tested classes, concentrating confirmations at the top of the anomaly ranking makes learning less efficient.