Statistical comparisons of time-series feature sets on classification tasks
2026-08-03 • Machine Learning
Machine Learning
AI summaryⓘ
The authors compared six popular open-source tools that extract features from single-variable time series to see how well they help classify data. They tested these tools on 124 different problems and found that most performed similarly, although one called tsfresh did slightly better overall. They also showed that for some problems, simple methods using basic statistics or Fourier analysis worked just as well. Their findings suggest it's important to look at how features perform on individual problems, as the mix of features can affect classification success.
time seriesfeature extractionclassificationtsfreshFourier coefficientsbenchmarkingunivariate dataspectral analysisquantilesopen-source software
Authors
Trent Henderson, Ben D. Fulcher
Abstract
In recent years, numerous open-source software libraries have been developed for computing sets of features from univariate time series. The type and number of features vary across these feature sets, which have been constructed with varying disciplinary perspectives on quantifying structure in time-series data. To date, the relative strengths and weaknesses of these feature sets on time-series classification problems remains largely unexplored. Here we aimed to understand the relative performance of six open-source feature sets and three baseline feature sets (based on distributional and/or basic spectral structure) across 124 univariate time-series classification problems using a normalization-based approach to problem-level benchmarking that better indexes the relative strengths and weaknesses of different algorithms compared to prior rank-based approaches. Despite their dramatic differences in size, composition, and computation time, we found that feature sets performed relatively similarly overall (85.3% of pairwise comparisons resulted in ties), with the largest feature set, tsfresh, exhibiting the strongest overall performance (29.03% wins across all pairwise comparisons against other feature sets). We also highlighted specific problems on which the specific composition of a given feature set gave it a substantial performance advantage or disadvantage, and problems where simple baselines comprised of Fourier coefficients and quantiles were sufficient to achieve strong performance. Our results demonstrate the need to consider problem-level performance when benchmarking time-series feature sets, and highlight the importance of feature make-up in driving relative classification performance.