TimeSeek improves time series analysis across tasks and domains
TimeSeek: Multi-Task and Multi-Domain Pretraining for Generalized Time Series Analytics
Machine Learning
Summary
Time series data records information over time, like temperature or stock prices, but analyzing it is tricky because patterns vary across different fields and tasks. The authors propose TimeSeek, a new model that learns from many types of time series data and can handle multiple tasks like predicting future values, detecting unusual events, or classifying patterns. TimeSeek uses special techniques to adapt to different kinds of data and tasks at once, making it more flexible and accurate. After training on lots of data, it performs better than previous methods on various time series problems, even with little or no data for the specific task.
What this means in practice
- •For industrial data engineers: Improve predictive maintenance by using one model trained across various sensors and tasks including fault detection and future value prediction.
- •For healthcare data analysts: Use a single adaptable model for detecting anomalies and forecasting patient vital signs from diverse medical time series data.
Authors
Peng Chen, Yihang Wang, Zhongwen Rao, Yang Shu, Yuying Qiu, Yuxuan Chen, Lujia Pan, Bin Yang, Chenjuan Guo
Abstract
Time series analytics has been widely applied across various domains. While existing time series foundation models (TSFMs) have demonstrated promising results, they still face two major challenges: 1) Most TSFMs mainly rely on stacking Transformer layers to increase parameter size, lacking the ability to adaptively model complex time series patterns from multi-domain data. 2) Current TSFMs are primarily pre-trained for specific tasks, e.g., forecasting or anomaly detection, and a few models support multiple tasks, but they fail to fully leverage task specificity and commonality, resulting in suboptimal performance. To address these challenges, we propose TimeSeek, a novel generalized time series analytics model that adaptively models time series patterns from multi-domain data and unifies multiple tasks in pre-training. Conceptually, TimeSeek is guided by selective information sharing under domain and task heterogeneity. From the perspective of multi-domain data, we propose a Multi-Domain Feature Extractor that adaptively captures temporal dependencies and channel correlations based on domain-specific characteristics, including progressive multi-scale modeling and adaptive slot channel modeling. From the perspective of the diversity of tasks, we propose multi-task collaborative training, which leverages task specificity and task commonality through task customization and cross-task training, effectively unifying multiple tasks in pre-training and better adapting to diverse downstream scenarios. After pre-training on large-scale time series data, TimeSeek achieves state-of-the-art performance in forecasting, anomaly detection, and classification tasks across both few-shot learning and zero-shot situations, together with favorable inference efficiency.