SVD-Based Typicality Maps for Out-of-Distribution Detection in Vision Transformers
2026-08-24 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors developed a way to better understand what a Vision Transformer (ViT) learns inside its layers by studying the geometry of its weight matrices. They break down weights using a math tool called Singular Value Decomposition and use this to create simple summaries of layer behavior called typicality maps. These maps help detect when the model sees data that is very different from what it was trained on, called Out-Of-Distribution detection. Their method works well on CIFAR-100 without needing extra training or examples of unusual data. The approach also includes two new scores to evaluate how well the model spots unfamiliar inputs.
Vision TransformerSingular Value Decompositionaffine layeractivationsclass-conditional densitytypicality scoreOut-Of-Distribution detectionPrototype Alignment ScoreMulti-Layer Soft VotingCIFAR-100
Authors
Aldo Sean Sartor, Leandro de Souza Rosa, Andriy Enttsel, Mauro Mangia, Riccardo Rovatti
Abstract
We present a method for analyzing the internal representations of Vision Transformers (ViTs) exploiting the geometry of their learned parameters. Each affine layer's weight matrix is factored via Singular Value Decomposition (SVD), and activations are projected onto the leading right singular vectors to obtain compact, layer-intrinsic representations. A class-conditional density model is then fitted at each layer, producing per-class \emph{typicality scores} that are stacked across depth into \emph{typicality maps}: two-dimensional summaries of how class-specific evidence evolves through the network. From these maps, we derive two post-hoc scores for Out-Of-Distribution (OOD) detection: a \emph{Prototype Alignment Score} (PAS), measuring agreement with class reference prototype patterns, and a \emph{Multi-Layer Soft Voting} (MLSV) score, capturing cross-layer consensus without stored prototypes. On ViT-B/16 fine-tuned on CIFAR-100, the proposed scores achieve competitive detection performance without retraining or OOD exposure.