Multi agent framework improves industrial anomaly detection accuracy near experts
Is In-Domain Training Enough for Fine-Grained Industrial Anomaly Understanding?
Computer Vision and Pattern Recognition
Summary
Finding tiny problems in industrial products is hard for one AI model to do all at once. The authors found training models only on industrial examples doesn't make them as good as human experts. They created a system named SiGMA that uses many AI 'agents' with different strengths and a special visual expert to work together. This team performs much better, almost as well as humans, without needing to retrain when new models join.
What this means in practice
- •For industrial quality control teams: Improve defect detection and localization in manufacturing using a multi-agent AI system that approaches human expert accuracy.
- •For visual inspection system developers: Integrate heterogeneous AI agents and reliability control to enhance fine-grained anomaly understanding without retraining for new models.
Authors
Xingwu Zhang, Duanyang Du, Huiling Zhu, Jiayue Dai, Yixiao Liu, Guozhi Liu, Zhihan Zhang, Zijun Long
Abstract
A single multimodal large language model (MLLM) struggles to excel simultaneously at detection, localization, description, and reasoning in multimodal industrial anomaly understanding (MM-IAU). We show that in-domain training does not close this gap. On MMAD, a widely adopted MM-IAU benchmark, trained specialists reach at most 75.5% accuracy in defect localization, against 92.3% for human experts, and even detect anomalies less accurately than their untrained base model. Meanwhile, different MLLMs offer complementary strengths but share this weakness in fine-grained perception, so combining them alone cannot remove it. We therefore propose SiGMA, a spatially grounded multi-agent framework that divides labor between heterogeneous MLLM agents and a dedicated visual defect expert. A multimodal searcher supplies industrial knowledge and normal references, the defect expert turns query-reference comparison into calibrated anomaly evidence, and a label-free reliability controller weighs each source by task-wise competence and query-level evidence quality. SiGMA reaches 85.2% average accuracy on MMAD, 4.0% above the strongest trained specialist and Gemini-2.5-Pro and within 1.5% of human experts. Even with three agents of at most 9B parameters, it reaches 84.4%, and new MLLMs join without retraining.