Cross-encoder method improves zero-shot detection of subtle image defects
Cross-Encoder Anchor Refinement for Zero-Shot Anomaly Detection
Computer Vision and Pattern Recognition
Summary
Detecting unusual or defective parts in images without specific training on those defects is very hard. The authors propose a new approach called Cross-Encoder Anchor Refinement that combines two types of image understanding to better find and pinpoint tiny problems. They tested this method on many industrial and medical image sets and found it better than previous techniques, especially at precisely locating defects. This approach shows how linking different visual processes early on can help spot subtle anomalies in new situations.
What this means in practice
- •For industrial quality control teams: Identify subtle defects in manufactured products without custom training for each defect type, improving automated quality checks in production.
- •For medical image analysts: Enhance automatic detection and localization of anomalies in medical images, aiding in diagnosis across varied unseen pathologies.
Authors
Kaisong Ji, Ziyang Xu, Chang Bian, Yu Zhou, Wenyu Liu, Xinggang Wang
Abstract
Zero-shot anomaly detection aims to transfer anomaly discrimination from auxiliary training domains to unseen categories while localizing subtle defects. A key challenge is to construct discriminative references that combine contextual representations with complementary local evidence. We present Cross-Encoder Anchor Refinement (CEAR), which connects two visual encoding paths for the same image at the anchor stage, before normal and anomalous prototypes are constructed. An independent convolutional encoder pretrained for industrial product recognition builds an image-specific memory shared across selected vision transformer (ViT) layers. Spatial anchors read from ViT patch features query this memory through cross-attention and undergo residual refinement. Image-conditioned normal and anomalous queries then gate anchor aggregation to construct prototypes for anomaly scoring. Across 13 industrial and medical datasets, CEAR achieves the best performance in 57 of 68 evaluations spanning image-level detection and pixel-level localization. Notably, CEAR surpasses all other state-of-the-art methods in 41 of 44 pixel-level comparisons. Across 11 localization datasets, mean gains in pixel-level maximum F1 score and average precision are 4.45 and 3.79 percentage points, respectively. This study shows how cross-encoder interaction at the anchor stage can integrate complementary visual representations for fine-grained anomaly localization across domains.