Self supervised pretraining improves some fault detection tasks but not all

Self-Supervised Pretraining for PHM: What Transfers, What Does Not, and What Federating It Costs

Artificial IntelligenceComputational Engineering, Finance, and ScienceMachine LearningPerformance

Summary

Predicting machine faults and remaining useful life can use computer models that learn from lots of unlabeled data, a technique called self-supervised pretraining. The authors tested this across 47 industrial datasets and found it helps some tasks like fault classification a lot, but does not improve remaining life predictions. They also compared learning with all data in one place versus many separate groups training together (federated learning), finding centralized training usually works better. The study shows that results depend strongly on which task, training settings, and evaluation methods are used. The authors provide their code and data tools for others to build on.

What this means in practice

Authors

José Sanz-Durán, David Solís-Martín

Abstract

Self-supervised pretraining has produced capable time-series foundation models, but its value for Prognostics and Health Management (PHM) has not been measured under leakage-controlled conditions. We measure it on a corpus of 47 public PHM datasets, harmonized to a single tensor contract and partitioned so that 11 datasets are withheld from pretraining in full. Transfer turns out to be task-dependent: a compact channel-independent Transformer encoder (801,808 parameters) pretrained with masked patch prediction raises fault-classification macro-F1 from 0.350 to 0.812 on CWRU and from 0.569 to 0.951 on HSG18, yet on CMAPSS remaining-useful-life estimation no pretrained checkpoint reliably improves upon a from-scratch baseline. We pretrain under two regimes---centralized, and simulated cross-silo federated with ten clients grouped by physical domain---holding architecture, objective, source data and optimization budget fixed. Two further findings are negative. At matched budget the centralized encoder leads by 15.5--20.2 macro-F1 points in four of four comparisons, although the comparison is confounded by a learning-rate schedule that favours the centralized arm and rests on a single pretraining run per regime; on RUL the two regimes are indistinguishable. And the intrinsic SSL loss is not a valid selection criterion: the centralized encoder more than halves the federated reconstruction loss without a corresponding gap on RUL, and a server-momentum sweep moves downstream validation macro-F1 by up to 0.29 while the SSL loss varies by 0.021. Transfer also hinges on the backbone learning rate: the same checkpoint yields 0.169 or 0.829 macro-F1 on CWRU depending on a single order of magnitude. We release the corpus pipeline, the protocol and the code.
View PDFOpen arXiv