Foundational vision models aligned once to enable clearer image recognition
Align Once to Explain: Feature Alignment for Scalable B-cosification of Foundational Vision Transformers
Computer Vision and Pattern Recognition
Summary
Vision models help computers understand images but are usually very hard to interpret. The paper introduces ALOE, a method that transforms these vision models so their decisions become easier to explain without needing new labels or much extra training. This transformation only needs to be done once and works well across different types of models and tasks. The aligned models keep their high accuracy while providing clearer insights about how they see images.
What this means in practice
- •For computer vision engineers: Use ALOE to convert large pretrained vision models into versions that provide clear, human-understandable explanations while keeping accuracy for classification and detection tasks.
- •For machine learning platform builders: Implement ALOE as a reusable feature alignment step so that interpretable vision backbones can be deployed efficiently across multiple downstream applications without retraining.
Authors
Raphael Maser, Siddhartha Gairola, Sukrut Rao, Bernt Schiele
Abstract
Foundational vision models have become the de facto standard for many vision tasks due to their strong performance. However, they are notoriously opaque and remain hard to interpret. We present ALOE (ALign Once to Explain), a one-time, label-free feature alignment approach that efficiently converts foundational vision models into inherently interpretable B-cos variants. Once aligned, the B-cos backbone is used as a drop-in replacement across several downstream tasks-amortizing the cost of interpretability. ALOE is robust across pre-training paradigms (supervised, self-supervised, vision-language) and can preserve teacher-level generalization with orders of magnitude less data than training from scratch. On classification, it strongly outperforms vanilla B-cosification (e.g., +10.5 p.p. top-1 on ImageNet for DINOv3 ViT-B/16), retains strong linear probing, k-NN, and zero-shot transfer competitive with foundational backbones (DINOv3, SigLIP2) across diverse downstream datasets. It also preserves spatially structured features useful for dense prediction, while yielding well-localized and highly human-interpretable explanations by design.