State-matched prediction improves uncertainty estimates in recurring conditions
State-Matched Sequence Conformal Prediction
Machine Learning
Summary
Predicting uncertainty in a sequence of data is tricky when the situation repeats but not in order. The authors propose a method called SMSCP that compares the current situation to similar past situations instead of just looking at recent data. This approach helps to pick better examples for estimating how uncertain predictions should be. Tests show SMSCP works better when certain states or conditions repeat after breaks, like in weather or bike-sharing demand. It often gives more accurate and reliable prediction intervals than other common methods.
What this means in practice
- •For energy forecasters: Improve confidence intervals for solar power predictions when weather states recur nonadjacently.
- •For urban mobility planners: Generate better demand forecasts with uncertainty estimates for bike sharing systems under varying conditions.
Authors
Ruihang Han, Li-Hsiang Lin
Abstract
Sequential conformal prediction needs to decide which past errors should calibrate uncertainty for the current target. Recency and online error feedback are useful when recent observations remain relevant, but the recent past can be a poor reference when comparable states recur at nonadjacent times, with the recurrence pattern generally unknown. We propose State-Matched Sequence Conformal Prediction (SMSCP), which constructs an explicit one-sided local-linear state from pre-target information, uses state similarity to select a matched reference set of past calibration scores, and then applies the ordinary unweighted conformal order statistic. In contrast to existing approaches, SMSCP uses state similarity to identify calibration observations relevant to the target score quantile. This motivates a theoretical analysis of whether nearby states have comparable target score quantiles under temporal dependence. We establish control of the resulting conformal cutoff and characterize predictive information not captured by the chosen state. Simulations show that state matching is most effective when score-relevant states recur across nonadjacent times, while revealing settings in which its advantage weakens or reverses. Across solar irradiance and bike-demand benchmarks, SMSCP improves interval score over recency, weighted, adaptive, and HopCPT comparators, and achieves coverage closer to the nominal 0.90 level than ResCP on five of six benchmarks.