Temper improves short term accuracy and calibrates time series forecasts
TEMPER: Temporal Encoder-Masked Probabilistic Ensemble Regressor for Time-Series Forecasting
Machine Learning
Summary
Forecasting future values in time series is hard because it's important to predict both accurate average values and how uncertain those predictions are. The authors introduce TEMPER, a new forecasting method that combines several techniques to produce predictions with calibrated uncertainty. Tested on simulated data with complex patterns, TEMPER shows good performance on short-term horizons and provides confidence intervals that are more reliable after calibration adjustments. Their study highlights that tuning predictions differently for each forecast time and improving calibration are key challenges ahead.
What this means in practice
- •For financial analysts: Generate calibrated uncertainty intervals for short-term financial time series to aid risk assessment and decision-making.
- •For energy demand planners: Produce probabilistic forecasts with improved accuracy and uncertainty calibration for horizon-specific load predictions in energy systems.
Tested on simulated data.
Authors
Giancarlo Vercellino
Abstract
Probabilistic forecasting requires accurate central predictions and calibrated uncertainty estimates. This paper presents TEMPER, the Temporal Encoder-Masked Probabilistic Ensemble Regressor, a univariate time-series forecasting algorithm that combines a temporal autoencoder, a differentiable masked neural decision forest, continuous ranked probability score (CRPS) training, and Gaussian-mixture post-processing. The R implementation is built on torch for R and returns horizon-wise density, distribution, quantile, and sampler functions. We evaluate TEMPER on three deterministic synthetic level series with trend, periodic, regime-switching, nonlinear-threshold, and heteroskedastic components. Across 96 rolling-origin forecasts at horizons t + 1, t + 5, t + 20, and t + 60, TEMPER obtains 2.824% mean CRPS normalized by origin level, 3.635% median absolute error, and 68.8% empirical 90% interval coverage after training with a 300-epoch cap and early-stopping patience of 100. A naive persistence bootstrap has the best aggregate CRPS, 2.763%, while TEMPER has the best median absolute error and the best CRPS at t+1 and t+5. The ablation study uses matched series-origin-horizon cells, horizon-wise CRPS deltas, endpoint sensitivity summaries, and a calibration-specific interval study. Relaxing the learned mask improves average CRPS by 0.472 percentage points on the ablation subset, mainly through long-horizon gains. A twofold interval inflation improves held-out coverage from 54.2% to 91.7% and gives the best 90% interval score among tested calibration rules. The results identify calibration, horizon-specific tuning, and component selection as the central research priorities.