Reinforcement learning improves decisions using predictions in uncertain environments
RL-PaO: Prediction as Action in Decision Making under Uncertainty
Machine Learning
Summary
Making good decisions when the future is uncertain is hard because predictions can be wrong and lead to costly choices. The authors introduce RL-PaO, a method that treats making predictions like taking actions in a game, learning from the results those predictions cause. This approach helps the system better align predictions with actual costs, improving decision quality without needing complex math inside optimization. They tested this on scheduling energy use based on past data and found it lowered costs by about 10%.
What this means in practice
- •For energy schedulers: Reduce operational costs in day-ahead energy planning by integrating prediction adjustments directly with decision outcomes.
- •For supply chain planners: Improve supply chain decisions by learning prediction actions that anticipate how errors impact downstream costs.
Authors
Jiahui Feng, Dafang Zhao, Zheng Chen, Zhengmao Li, Lingwei Zhu
Abstract
Decision-making under uncertainty often relies on predicted parameters, yet accurate prediction does not necessarily lead to good operational decisions. Aligning prediction with downstream optimization requires learning from the consequences of the decisions those predictions induce. We introduce RL-PaO, a reinforcement learning framework that integrates system formulation, optimization, and decision execution into a single environment. This yields a Markov decision process in which prediction is regarded as action: it shifts the environment to produce subsequent context and reward that explicitly aligns prediction error with realized cost, and learning the optimal policy does not require differentiating through the black-box solver. We evaluate RL-PaO on day-ahead energy scheduling using real historical data. On the test year, RL-PaO achieves the lowest annual cost among the non-oracle baselines, achieving on average $10\%$ cost reduction. Moreover, RL-PaO is capable of further analyses to provide strong interpretability both from the policy evolution perspective and the cost-accuracy trade-off.