Point clouds prove more reliable than pixels for zero-shot anomaly detection

When Point Clouds Outperform Pixels: Rethinking Zero-Shot Multimodal Anomaly Detection

Computer Vision and Pattern Recognition

Summary

Detecting unusual or faulty parts in things like machines or objects is often done by looking at their pictures, but these pictures can sometimes be misleading when used without prior training. The authors found that 3D point cloud data, which capture the shape and geometry of objects, are more reliable than regular images in spotting anomalies especially when the system has never seen that category before. Based on this, they built a new method called WOOPS that pays more attention to 3D data and adjusts how much it trusts images depending on the situation. Their tests showed this approach detects anomalies better and also helps improve image-only methods.

What this means in practice

  • For industrial inspection teams: Improve detection of defects on products without prior examples by using reliable 3D shape data alongside images to reduce false alarms during quality control.
  • For robotics software developers: Create more dependable anomaly recognition in new environments by calibrating sensor inputs based on their trustworthiness to handle unknown object categories.

Authors

Chenglin Ye, Lupeng Liu, Dongbo Yu, Jun Xiao, Yunbiao Wang

Abstract

Zero-shot multimodal anomaly detection commonly assumes that RGB and point cloud modalities are equally reliable and can contribute uniformly to anomaly localization. We challenge this assumption. Using a set of recently proposed stringent metrics that penalize false anomaly responses in normal regions, we find that point clouds are substantially more reliable than RGB under zero-shot category shift. Motivated by this observation, we propose WOOPS (\textbf{W}hen P\textbf{o}int Cl\textbf{o}uds Out\textbf{p}erform Pixel\textbf{s}), a reliability-aware zero-shot multimodal anomaly detection framework. To strengthen the more reliable geometric modality, we design a Multi-view Information Decoupling module to suppress heterogeneous information from multi-view point cloud projections and enhance point cloud feature quality. To avoid unconditional fusion, we further introduce a Modality Reliability Calibration module to adaptively calibrate modality contributions according to their reliability. Extensive experiments show that our method achieves the best or competitive performance under the new metrics in both unimodal and multimodal settings. Further analysis demonstrates that point cloud information also improves RGB-only inference, while ablations verify the effectiveness of both modules. Code will be released upon acceptance.
View PDFOpen arXiv