Time series segmentation benchmark compares 26 algorithms across many datasets
TSB-SEG: A Systematic Time-Series Segmentation Benchmark
Machine Learning
Summary
Time series segmentation means finding important points or states in sequences of data over time. The authors saw that two groups—statisticians and machine learners—usually study this problem differently and separately. They created TSB-SEG, a big benchmark with many datasets and algorithms to fairly compare these approaches together. Their study shows no single method works best in all cases, tuning parameters carefully helps a lot, and the type of data matters when choosing a method.
What this means in practice
- •For data engineers: Choose and tune time series segmentation methods suited to diverse datasets using a unified benchmark and open-source tools.
- •For monitoring system developers: Evaluate multiple segmentation algorithms on varied data to improve fault detection and diagnostics in complex real-world systems.
Authors
Félix Chavelli, Arik Ermshaus, Fan Yang, Patrick Schäfer, John Paparrizos, Paul Boniol
Abstract
Time-series segmentation, studied as either Change Point Detection (CPD) or State Detection (SD), underpins a broad range of monitoring and diagnostic applications. However, the field suffers from a persistent divide between the statistical community (focusing on CPD) and the machine learning and data mining communities (focusing on SD). Consequently, the literature remains fragmented: evaluations typically cover narrow methodological families, datasets are often small or homogeneous, and CPD and SD are rarely evaluated together. We address these gaps with TSB-SEG, a unified benchmark comprising 592 univariate and multivariate time series from 8 heterogeneous datasets and surveying 26 algorithms that span five decades of progress. To ensure a fair comparison, we introduce a two-pool design: a scalable main pool of tractable detectors used for the aggregate ranking, and an extended-scope pool for computationally intensive methods. All methods are unified under tsseg, an open-source library to ensure full reproducibility. Our systematic evaluation yields three insights: (a) no single detector dominates across domains, and per-dataset hyperparameter tuning remains decisive for performance, (b) CPD accuracy is a useful, though imperfect, proxy for SD accuracy, and (c) data characteristics should guide method selection. We provide actionable guidance based on data properties, error tolerances, and tuning budgets, identify impactful hyperparameters, and characterize algorithm robustness to these choices.