Multiproduct pipeline scheduling improved by separate deep reinforcement learning actions
Efficient Detailed Scheduling of Multiproduct Pipelines using Action-Decoupled Deep Reinforcement Learning
Machine Learning
Summary
Scheduling different products in a shared pipeline is tricky because the flow must follow complex water movement rules and meet delivery demands. The authors developed a new kind of AI method that splits decision-making into parts — choosing how long to operate and at what flow rate separately — to better handle these rules. This approach creates accurate and stable schedules much faster than traditional optimization methods. They tested it on a real pipeline in China and found it almost matches exact solutions while running more efficiently.
What this means in practice
- •For pipeline operation teams: Produce precise and efficient delivery schedules for multiproduct pipelines that respect complex hydraulic flow rules.
- •For industrial scheduling software developers: Integrate an action-decoupled reinforcement learning framework to improve nonlinear system scheduling under hybrid action constraints.
Authors
Kai Shao, Ling Jian, Liming Liu, Serguey V. Bogomolov, Xinmin Wang
Abstract
Multiproduct pipelines transport refined products to delivery stations in sequential batches, and their detailed scheduling must meet market demand while respecting nonlinear hydraulic constraints. Retaining the full hydraulic nonlinearity yields a mixed-integer nonlinear program that is computationally intractable, while the linearization commonly adopted for solver-based optimization degrades hydraulic accuracy without resolving the heavy computational burden. This paper develops an action-decoupled proximal policy optimization (PPO) framework that efficiently produces detailed schedules while preserving hydraulic fidelity. The scheduling process is cast as a semi-Markov decision process (SMDP) with flexible action durations. To handle the discrete-continuous hybrid action space, a cascaded dual-actor architecture decouples the discrete operating duration from the continuous flow decisions, eliminating the gradient interference that destabilizes conventional coupled-action policies and thereby yielding high-quality schedules far more efficiently. Validated on a real-world multiproduct pipeline in China and benchmarked against a single-actor baseline and a linearized MILP solved by Gurobi, the proposed framework achieves delivery accuracy close to the exact optimum while substantially improving flow rate stability and substantially reducing solution time.