Normality-Preserving Continual Industrial Anomaly Detection via Orthogonal LoRA Banks
2026-06-01 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors address the problem of continual industrial anomaly detection, where models forget old normal behaviors when learning new ones. They propose a method that keeps old knowledge safe by freezing previous model parts and making new learning happen in spaces that don't interfere with old knowledge. Their approach also grows the model's capacity only when needed. Tests show their method works better than previous ones at spotting anomalies over long sequences of categories.
Continual learningAnomaly detectionDiffusion modelsCatastrophic forgettingLow-rank adaptation (LoRA)U-NetOrthogonal subspaceResidual updatesMVTec datasetVisA dataset
Authors
Weibai Fang, Haijun Che, Feiyang Ren, Qiancheng Lao
Abstract
Continual industrial anomaly detection with diffusion models suffers from historical normality prior drift and catastrophic forgetting. Existing continual diffusion methods preserve previous knowledge through replay or constrained optimization, but they lack an explicit mechanism for isolating and protecting category-specific normality priors during sequential adaptation. Although low-rank adaptation provides modular residual updates, standard LoRA neither freezes historical normality subspaces nor prevents new adapters from interfering with previous ones. To address this issue, we propose a normality-preserving continual anomaly detection framework based on two modules: History Frozen Orthogonal LoRA Bank (HF-OLB) and Hierarchical Novelty Adaptive Bank Growth module (HNABG). HF-OLB freezes both the pre-trained U-Net backbone and the learned LoRA banks, and constrains new task-specific normality residuals to the orthogonal complement of historical LoRA subspaces. HNABG further allocates layer-dependent residual capacity and expands the bank only when the residual normality novelty exceeds the expressive capacity of existing banks. Extensive experiments on MVTec and VisA demonstrate the effectiveness of the proposed method. On the challenging VisA 2x6 setting, our method achieves 83.6/91.8 image and pixel level A-AUROC with 3.8/3.9 FM, improving pixel level A-AUROC over the state of the art by 3.2 points while reducing pixel level FM by 1.3. These results show that our method effectively preserves historical normality priors in long horizon continual category sequences.