Fine grained aircraft detector confusion sources diagnosed and reduced
Not All Confusion Is Equal: A Source-Aware Uncertainty Diagnosis for Fine-Grained Aircraft Detection
Computer Vision and Pattern RecognitionMachine Learning
Summary
Detecting specific types of aircraft can be tricky because models sometimes confuse similar classes without a clear reason. The authors show that confusion isn’t all the same—it comes from different identifiable sources that need different fixes. They created a tool called A2E2 that breaks down confusion into four categories based on what causes it. This helps them find targeted ways to improve the model, like fixing data labels or gathering more data. Their approach was tested and shown to reduce specific types of confusion without affecting others.
What this means in practice
- •For computer vision engineers: Diagnose and address distinct confusion sources in fine-grained object detectors to improve aircraft model accuracy with targeted interventions.
- •For data annotation teams: Identify and prioritize confusing subclasses needing relabeling or more data in large-scale aircraft detection datasets for better training quality.
Tested on one dataset.
Authors
Hai Huang, Helmut Mayer
Abstract
Fine-grained object detectors are commonly evaluated with confusion matrices, which show where the model is confused but not why, nor whether the confusion can be reduced. We argue that confusion can be attributed to distinct, separable sources, each quantitatively measurable, turning a passive measurement into actionable guidance. We present $A^2E^2$, a diagnostic tool that decomposes the sources of confusion along two axes, $\{$aleatoric, epistemic$\} \times \{$within-class, between-class$\}$, giving a $2\times2$ taxonomy that enumerates the source types. Each quadrant is measured by its own quantity, computed in one of three places (input geometry, output-space disagreement, and the bias-parameter posterior), so the two epistemic sources are separated by construction rather than by an empirical correlation. On fine-grained aircraft detection, the four quadrants become four named sources with their own remedy verdict: affinity (geometric similarity, irreducible from size alone), heterogeneity (geometrically heterogeneous sub-variants, pointing to re-labeling rather than more data), contested (an insufficiently trained but learnable boundary, improvable), and collapsed (a class starved of data, reducible). After attributing the confusion to a specific reducible source, we apply a targeted intervention and verify experimentally that it reduces the diagnosed source specifically while leaving the irreducible sources unchanged. $A^2E^2$ thus turns confusion measurement into a concrete, validatable and actionable "diagnosis" in which the same off-diagonal mass can carry opposite causes and opposite remedies. We also state this framework's limits, including which sources are only partially identifiable on this specific dataset and why.