Online Estimation of Dynamic Origin-Destination Matrices Using Reinforcement Learning with Link-Flow Propagation Guidance
2026-08-31 • Machine Learning
Machine LearningArtificial Intelligence
AI summaryⓘ
The authors developed a new method called LFPG-RL to better estimate how many trips start and end in different places over time using real-time traffic data. Their approach improves existing methods by guiding a reinforcement learning algorithm to focus on the parts of the demand that most affect traffic errors. This allows the model to quickly and accurately update demand estimates with only one computation step when new data arrives. They tested this on traffic data from Melbourne and found it to be more accurate and efficient than previous methods. Overall, their work helps make online traffic demand estimation faster and more reliable.
origin-destination (OD) matrixdynamic OD estimation (DODE)link-flow trajectoriesreinforcement learning (RL)proximal policy optimization (PPO)link transmission modelstochastic dynamic network loading (DNL)traffic demand calibrationadvantage shapingRMSE
Authors
Donggyu Min, Dong-Kyu Kim
Abstract
Online dynamic origin-destination (OD) matrix estimation (DODE) calibrates time-dependent OD demand to reproduce observed link-flow trajectories. In online, OD demand should be estimated from current observations and propagated network states while subsequent observations and stochastic dynamic network loading (DNL) outcomes remain uncertain. Recently, reinforcement learning (RL) has emerged as a promising alternative, reducing computational burden by replacing iterative algorithms while being applicable to stochastic environments. However, because the policy is trained offline and deployed online, it must handle varying target link-flow trajectories; since each target trajectory defines the link-flow error used in the reward, the same OD demand vector can require different adjustments, making conventional scalar feedback ambiguous. To address this gap, this study proposes LFPG-RL, which integrates link-flow propagation guidance (LFPG) into proximal policy optimization (PPO). LFPG combines link-flow error sensitivities with the contribution of each OD-time demand component to simulated link flows, transforming aggregate mismatch into OD-specific advantage shaping for PPO actor updates. At deployment, the policy requires only a single forward pass. LFPG-RL is developed and evaluated on 250 weekday trajectories of 15-min link-flow data from a Melbourne arterial network modeled by a link transmission model with stochastic route choice. On held-out trajectories, LFPG-RL achieved an RMSE of 4.69, MAPE of 20.15%, and Pearson correlation of 0.995. These results support the contention that our method is a more efficient and accurate online OD demand calibration method compared to existing ones.