Pretraining improves industrial sensor data analysis with limited labels
Cross-Domain Joint-Embedding Pretraining For Heterogeneous Industrial Time Series
Machine LearningArtificial Intelligence
Summary
Many industrial machines generate large amounts of sensor data, but often there are only a few labeled examples for training AI models. The authors study a way to teach a single AI encoder to understand time series data from different types of machines, like batteries and bearings, using unlabeled data. This pretraining helps the models perform better when they only have a small amount of labeled data to learn from. Their approach reduces errors and improves fault diagnosis across various industrial tasks.
What this means in practice
- •For industrial machine learning teams: Reuse pretrained encoders on sensor data from varied machinery to improve predictive maintenance model accuracy with few labeled examples.
- •For manufacturing process engineers: Use pretrained time series models to detect faults or predict wear in manufacturing equipment across different process types with limited labeled data.
Authors
Jonathan Adam Rico, Nagarajan Raghavan, Senthilnath Jayavelu
Abstract
Industrial predictive maintenance and manufacturing tasks frequently contain abundant sensor measurements, but with limited labeled data. We investigate cross-domain self-supervised pretraining using joint-embedding predictive architecture for heterogeneous industrial time series (JEPA-HITS), which learns one encoder from various domains such as battery, semiconductor chemical mechanical planarization (CMP), computer numerical control (CNC) machining, and bearing while preserving dataset-specific temporal scales. We evaluate representation reuse on regression and classification problems under reduced-label downstream training, and leave-one-domain-out (LODO) pretraining. Compared with the same architecture initialized randomly, LODO pretraining reduces RMSE from 10.72 to 7.95 on battery remaining useful life (RUL), 4.34 to 3.89 on CMP, and 13.53 to 11.72 on tool wear regression, while improving bearing fault diagnosis macro-F1 from 0.547 to 0.737. Overall results suggest that unlabeled time series measurements from heterogeneous industrial processes can provide reusable representations for downstream learning with limited labeled data.