Electricity market forecasts can leak future data unknowingly
The Hidden Look-Ahead: Point-in-Time Leakage in Electricity Market Data
Machine LearningComputational Engineering, Finance, and Science
Summary
Forecasts of electricity prices often claim to use only the information available at the time decisions are made. However, the authors show that timing issues in how weather data and market times are recorded can cause models to accidentally use data from the future. They studied many weather forecast runs and electricity market auctions and found that in most cases, the chosen weather forecast wasn't actually available when the market decision had to be made. This hidden data leakage can affect the accuracy of predictions, sometimes improving or worsening it, so it needs careful measurement and monitoring.
What this means in practice
- •For electricity market analysts: Detect and prevent future data leaks in forecasting pipelines to produce more realistic electricity price predictions at auction times.
- •For weather data engineers: Implement timestamp-based auditing protocols to ensure published weather forecasts align correctly with application deadlines and avoid hidden look-ahead biases.
Authors
Abdelkrim Alfalah
Abstract
Empirical studies of electricity prices routinely state that their models use only information available at the time of the trading decision. For weather-driven forecasts drawn from commercial archives, this condition is harder to meet than it appears, and whether it holds can change over time. We identify three timing pitfalls. A forecast run is labelled by the issue time of its weather model, but becomes available hours later and keeps changing after publication. Auction gates follow local clock time, while archives use UTC. Outturn series are revised after publication without earlier values being kept. Using the timestamps of 128,656 forecast runs from ten weather-model configurations, matched to the day-ahead and first intraday auction gates in Great Britain over 2022-2026, we find that selecting "the latest run issued before the gate" picks a run not yet published at the gate on more than 99% of delivery days for six of the ten configurations at one of the two gates. For GFS at the day-ahead gate, the share is 0.6% in winter and 99.8% in summer. The exposure also shifts as publication delays drift, and an action margin of a few minutes can cost a full six-hour forecast cycle. The wrongly selected inputs differ materially from those available at the gate, but their apparent accuracy gain ranges from -9% to +24% across variables, models, gates and seasons, so its effect cannot be assumed and must be measured. We argue that point-in-time correctness is a dynamic property of the data pipeline, and propose a protocol that audits it from timestamps before any model is fitted.