Vision language models detect anomalies without extra training
RADIOlogy: A Single-Pass Approach to Training-Free Zero-Shot Anomaly Detection
Computer Vision and Pattern Recognition
Summary
Detecting small or subtle problems in images usually needs lots of training or slow scanning techniques. The authors found that certain vision-language AI models have a surprising ability to spot anomalies right away without any training. They created RADIOlogy, a fast method that uses this ability to detect and localize anomalies by comparing images to normal and abnormal prompts. This method works well even in specialized fields like medical imaging without complicated extra steps.
What this means in practice
- •For medical imaging teams: Detect anomalies in medical scans quickly without specialized training datasets using RADIOlogy’s vision-language model technique.
- •For quality control engineers: Implement a swift anomaly detection system for industrial images that doesn’t need window scanning or extra training data.
Authors
Joaquín Figueira, Giacomo D'Amicantonio, Egor Bondarau
Abstract
Current zero-shot anomaly detection methods often rely on complex prompt-learning techniques or computationally expensive sliding-window strategies to achieve fine-grained visual alignment to detect small-scale anomalies. This reliance stems from the fundamental limitation of standard CLIP architectures, where [CLS] token attention and spatial patch pooling struggle to capture subtle, localized anomaly features. While prompt-learning approaches aim to bridge this gap, they typically require a large number of ground-truth segmentation masks from auxiliary datasets, which are often unavailable in specialized domains. In this work, we show an emerging property of multi-head attention pooling in vision-language models: remarkable zero-shot anomaly localization and detection right out of the box. We translate this property into a single-pass anomaly detection technique, RADIOlogy, that does not require dense, windowed execution and significantly outperforms other training-free baselines. Instead, the method relies on a simple contrastive scoring approach that compares the average similarity between vision features and a suite of normal and abnormal prompts. Furthermore, we leverage the agglomerative features of RADIO models to enhance pixel-level anomaly segmentation using a dense anomaly refinement pipeline. Finally, we show how the vision-language family of training-free zero-shot anomaly detection techniques can be applied successfully to the medical domain with minimal modifications. Code is available at https://github.com/jacomof/RADIOlogy.