Time series models show consistent patterns of poor uncertainty estimates
Evaluating Uncertainty Calibration in Probabilistic Time Series Foundation Models
Machine LearningArtificial Intelligence
Summary
Predicting future data points with an idea of how uncertain those predictions are is important for many applications. This paper looks at how well large time series forecasting models estimate their own uncertainty over various time horizons. The authors found that these models often misjudge how uncertain their forecasts are in ways that simple checks can miss. They also explored methods that try to fix these misjudgments and identified when they work and when problems remain. Their findings help practitioners understand and improve uncertainty estimates in these forecasting tools.
What this means in practice
- •For financial analysts: Improve risk assessment in financial forecasts by better evaluating and adjusting time series models' uncertainty estimates over different time horizons.
- •For energy demand planners: Refine probabilistic predictions of energy consumption with calibrated uncertainty to support more reliable planning and operations.
Authors
Thomas Decker, Volker Tresp
Abstract
Time series foundation models (TSFMs) are increasingly used for probabilistic forecasting across diverse domains, yet the reliability of their uncertainty estimates remains poorly understood. In this work, we study uncertainty calibration in probabilistic TSFMs from a broader methodological and empirical perspective. We introduce a taxonomy of how TSFMs represent predictive uncertainty and develop a fine-grained evaluation framework for calibration in multi-horizon forecasting that distinguishes pooled, horizon-wise, and stronger conditional notions of calibration. Using this framework, we conduct a comprehensive empirical evaluation of popular TSFMs across diverse datasets and show that current models exhibit distinct and systematic patterns of miscalibration that standard metrics often obscure. We further analyze common post-hoc recalibration methods and characterize which forms of calibration they can improve and which tend to persist. Our results provide practical guidance for evaluating and improving uncertainty calibration in TSFMs and highlight important limitations of current practice.