Vision language models improve zero shot anomaly detection accuracy
Proximity-CLIP: Text-Guided Semantic Proximity Learning for Zero-Shot Anomaly Detection
Computer Vision and Pattern Recognition
Summary
Detecting unusual or defective items in images without prior training is difficult because normal and abnormal features often look similar. The authors propose Proximity-CLIP, which adjusts how text descriptions relate to visual data to better separate normal from abnormal features. This method also focuses on small defective areas using a special querying mechanism and preserves smooth visual structure learned from before. Their approach performs better than existing methods on several tests with only minor changes to the model.
What this means in practice
- •For quality control engineers: Identify defects in manufacturing images without tailored training data by using calibrated text-guided visual models that focus on subtle anomalies.
- •For security surveillance teams: Automatically detect unusual objects or events in camera feeds leveraging text-vision model alignment to spot anomalies promptly and accurately.
Authors
Manwen Yang, Leqian Ding, Yu Guo, Fei Wang
Abstract
Vision-language models offer a promising approach for zero-shot anomaly detection (ZSAD). However, due to object-centric bias, normal and anomalous text prototypes exhibit a high semantic overlap. While enforcing strict orthogonality between them improves discriminability, mapping highly contiguous visual inputs onto drastically orthogonal prototypes introduces a geometric dilemma, disrupting the pre-trained structural continuity. To address this problem, we propose Proximity-CLIP, a framework that visually calibrates the semantic margin to guide visual adaptation. First, we introduce a visually-calibrated semantic proximity learning mechanism that uses a bounded dynamic regularization to learn an appropriate semantic margin, ensuring discriminative separation while preserving structural alignment. Second, we design an Anomaly Query Module (AQM) driven by these text priors. Using the calibrated anomalous prototype as a semantic query, the AQM actively retrieves localized defect cues from contextual visual patches, mitigating the dilution of subtle anomalies during global pooling. Extensive experiments demonstrate that Proximity-CLIP outperforms current state-of-the-art methods across multiple ZSAD benchmarks with minimal architectural modifications.