Fine grained visual classification improved with constrained attention and distillation
CARE: Constrained Attention Refinement for Fine-Grained Visual Classification via Teacher-Student Distillation
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Fine-grained visual classification means telling apart very similar-looking things, like bird species or dog breeds, by spotting small details. The authors propose a new way to train computer vision models to focus on these subtle features while also explaining their decisions more clearly. They use a system where a 'teacher' model helps a 'student' model learn by sharing knowledge about important parts of images. This method improves how well the model classifies images and highlights meaningful features that support its choices.
What this means in practice
- •For computer vision engineers: Develop more interpretable fine-grained classifiers for identifying subtle differences in image categories like animals and vehicles under frozen pretrained backbones.
- •For machine learning model auditors: Assess model explanations by validating that top attention regions correspond to meaningful image features relevant to classification decisions.
Authors
Ruibo Wen, Hang Shao, Yiming Lei
Abstract
Fine-grained visual classification requires models to recognize subtle local traits while exposing the visual evidence behind their predictions. Class-specific attention pathways provide a natural basis for interpretable recognition, but their constrained prediction structure limits discriminative capacity and underuses intermediate representations from strong pretrained backbones. To address this problem, we propose CARE, a constrained attention refinement framework for interpretable fine-grained recognition via teacher-student distillation. CARE keeps the final prediction and explanation within a class-specific attention student, while introducing a training-only auxiliary query teacher that reads selected intermediate DINOv2 layers with learnable queries. The teacher fuses multi-level representations and transfers logit-standardized class-discriminative knowledge to the student. To further refine the explanation pathway, we design diversity and sparsity terms to regularize student attention heads, reducing redundancy and encouraging compact trait localization. Experiments on CUB, Oxford-IIIT Pet, Stanford Dogs, and Stanford Cars show that CARE achieves strong classification performance under an interpretable frozen-backbone setting, reaching 78.5% Top-1 accuracy on CUB. Faithfulness analysis with insertion and deletion metrics further indicates that the top-ranked attention regions retain class-relevant evidence for explanation.