Trajectory-Initialized Neural Double Q-Routing for Large-Scale Overhead Hoist Transport Systems
2026-08-31 • Machine Learning
Machine LearningArtificial Intelligence
AI summaryⓘ
The authors study how fleets of overhead hoist transport robots, which share limited pathways in factories, can get delayed by traffic and congestion. They find that traditional routing methods either don't adapt well to changing traffic or learn too slowly. To fix this, they propose a neural network-based routing method that learns smarter paths by sharing information across routes and improving itself over time. Their method generally speeds up task completion compared to older techniques, especially in larger fleets, and performs better after offline training. Overall, the authors show improved efficiency in complex robot routing scenarios.
overhead hoist transport (OHT)routing algorithmsQ-routingneural networksDouble Q-learningcongestion managementmaterial handling systemsshortest-path routingreinforcement learningsimulation
Authors
Cheng Gu, Qiusheng Zhao, Anbang Liu, Shaochong Lin, Max Z. J. Shen
Abstract
Large-scale industrial robot fleets share constrained physical infrastructure, making vehicle travel times dependent on safety separation, intersection access, downstream blocking, and station contention. We study this problem in overhead hoist transport (OHT) systems, a representative ceiling-mounted material-handling system used in semiconductor fabs. Static shortest-path routing cannot account for these time-varying traffic costs, whereas tabular Q-routing adapts online but learns each destination--node--action value independently, limiting information sharing across sparsely visited routing contexts and making startup behavior sensitive to inaccurate value estimates. We propose Neural Double Q-routing, which replaces destination-indexed tables with a shared state--action value network. The network is warm-started through return-to-go regression on mixed simulator-generated routing trajectories and then refined online using Double-Q updates, local congestion correction, and event-stratified structured replay. Across nine matched fleet-size--arrival-rate settings with 100, 150, and 200 OHTs, the proposed framework reduces mean completion time relative to tabular Double Q-routing by $0.8\%$--$8.8\%$. It achieves the lowest mean completion time among all compared methods in the six 150- and 200-OHT settings, whereas Dijkstra remains best in the three 100-OHT settings. Completed-task counts remain within $1\%$ of tabular Double Q-routing in eight of nine settings, and 95th-percentile completion time decreases in eight settings. In two matched startup scenarios, offline initialization increases the number of completed tasks by up to $23\%$ and reduces tail completion time by up to $15\%$.