DeliveryGym environment helps agents plan courier work shifts better
DeliveryGym: An RL Environment for Long-Horizon Embodied Agent Planning with Adaptive Curriculum
Machine Learning
Summary
Planning deliveries is hard because each decision uses up time, energy, or money that affects future tasks. The authors made DeliveryGym, a 3D simulation where agents learn to plan whole courier shifts by seeing the long-term results of their actions. DeliveryGym adjusts its challenges based on where agents struggle to help them improve. Tests showed training inside DeliveryGym lets agents make more money and coordinate deliveries better over time.
What this means in practice
- •For robotics teams: Train delivery robots in simulations that track resource use across whole work shifts for better real-world planning.
- •For logistics software developers: Build systems that optimize courier schedules by learning from entire delivery sequences rather than isolated tasks.
Authors
Haoqiang Kang, Yiming Zhang, Yiyang Guo, Chuying Li, Jianzhi Shen, Tianruo Rose Xu, Xiaokang Ye, Lianhui Qin
Abstract
Executable environments enable LLM agents to learn from the consequences of their actions. For embodied agents, those consequences extend beyond whether the current task succeeds: completing a delivery can consume the time, energy, or money needed for later work. Learning to plan therefore requires environments that preserve these dependencies and turn them into feedback across a complete trajectory. We introduce DeliveryGym, a 3D environment for evaluating and training agents on continuous courier shifts. It couples multimodal tool interaction with persistent world dynamics and computes trajectory rewards from simulator events, making the costs of an agent's decisions available for reinforcement learning (RL). The environment also adapts future training shifts to the policy's observed weaknesses while keeping evaluation fixed. Across six models and 13 city maps, evaluation exposes a gap between reliably executing assigned deliveries and choosing and sequencing work over a shift. On the fixed test suite, RL improves Qwen3-VL-4B's net income by 54.3%, showing that learning from complete shifts improves performance under these coupled constraints. Adapting the training environment improves test income by 16.5% over uniform sampling at the same rollout budget, indicating that which situations an agent practices also matters. DeliveryGym provides an executable setting for studying how agents learn to coordinate deliveries and preserve resources for later orders within an episode.