Shortcut Trajectory Planning for Efficient Offline Reinforcement Learning
2026-07-10 • Machine Learning
Machine LearningArtificial IntelligenceRobotics
AI summaryⓘ
The authors present Shortcut Trajectory Planning (STP), a new method for planning actions in offline reinforcement learning that speeds up the process of generating action sequences. Unlike previous methods which require many steps or complicated two-phase training, STP uses a simpler single-stage approach and can adjust how many steps it plans ahead. It also uses a special evaluator to pick the best plans by checking their feasibility. Tests on various control tasks show that STP works well while making training faster and less complex.
offline reinforcement learningtrajectory planningdiffusion modelsconsistency modelsdenoising processteacher-student distillationmodel-based RLtrajectory generationcritic networksD4RL benchmarks
Authors
Guanquan Wang, Yoshimasa Tsuruoka
Abstract
Diffusion-based trajectory planners have shown strong performance in offline reinforcement learning, but their iterative denoising process often incurs high inference cost. Consistency-based planners reduce the number of sampling steps, yet they typically rely on a two-stage teacher--student distillation pipeline that increases training cost and may introduce instability. We propose Shortcut Trajectory Planning (STP), an offline model-based reinforcement learning framework that incorporates shortcut models as efficient trajectory generators. STP trains a conditional shortcut trajectory model in a single stage, supports adjustable one-step and few-step inference through step-size conditioning, and selects candidate plans using a critic augmented with feasibility-aware correction. Across standard D4RL benchmarks, including locomotion, navigation, manipulation, and dexterous control tasks, STP achieves strong performance while simplifying the training pipeline for fast generative planning.