Time series foundation models balance accuracy and reliability differently
Evaluating Accuracy and Probabilistic Reliability of Zero-Shot Time Series Foundation Models
Machine LearningArtificial Intelligence
Summary
Forecasting future events using time-based data is hard, and new time series foundation models (TSFMs) promise to do this without extra training for each task. The authors compared six of these models on energy, traffic, and financial data. They found that while TSFMs generally predict better than traditional methods and supervised deep learning models, there is a trade-off: some models are more accurate but less reliable in expressing uncertainty, while others are better calibrated but less accurate. This study helps developers choose the right model depending on whether they prioritize exact predictions or trustworthy uncertainty estimates.
What this means in practice
- •For energy system planners: Use TSFMs with xLSTM architecture to obtain well-calibrated uncertainty estimates for energy demand forecasting.
- •For financial risk analysts: Employ patch-based transformer TSFMs to gain competitive point forecasts while acknowledging limitations in uncertainty calibration at long horizons.
Authors
Panagiotis Michael, Moysis Symeonides, Demetris Trihinas
Abstract
Time Series Foundation Models (TSFMs) promise a paradigm shift toward zero-shot forecasting by eliminating task-specific training. However, existing works often overlook trade-offs between predictive accuracy and probabilistic calibration. This paper presents a benchmark study of six TSFMs evaluated on energy, traffic, and financial datasets. We contrast their performance against statistical baselines and a supervised DL model. The study reveals that while TSFMs outperform statistical methods and supervised models, they are subject to a fundamental trade-off between point accuracy and probabilistic reliability. Specifically, xLSTM architectures provide robust probabilistic calibration across horizons. In contrast, patch-based transformers offer competitive accuracy but face calibration issues at long horizons, while transformer-based models exhibit context saturation points for optimal zero-shot reasoning. These findings offer evidence-based guidance for balancing generalization and uncertainty quantification in real-world deployments.