Comparing four reinforcement learning methods for safe autonomous driving
Comprehensive Comparisons of Reinforcement Learning Algorithms in Autonomous Driving
Machine Learning
Summary
Making self-driving cars decide safely and well is complicated, so the authors studied four computer learning methods for this task at an intersection without traffic lights. They measured how good and safe the decisions were, how steady the training process was, and how much time and data each method needed. One method called RCPI was the safest and most steady but needed lots of training data. Another, D-DDPG, used data efficiently but was unstable during training. These findings help choose the right method based on available information, training budget, and stability needs.
What this means in practice
- •For autonomous vehicle engineers: Select the most appropriate RL algorithm for vehicle decision-making based on data availability and training constraints.
- •For industrial robot developers: Inform choices of reinforcement learning methods for robot decision tasks requiring stable training and safety assurance.
Authors
Dechen Xie, Mushuang Liu
Abstract
This paper presents a comprehensive comparison of four representative reinforcement learning (RL) methods in autonomous vehicle (AV) decision-making, including: Fitted Value Iteration (FVI), Rollout Classification Policy Iteration (RCPI), Deep Q-Network (DQN), and a discrete-action adaptation of Deep Deterministic Policy Gradient (D-DDPG). The four methods are applied to the same AV decision-making problem at an unsignalized intersection and are compared in terms of optimality and safety, training stability, and training and decision cost. RCPI is the closest to a reference solution and has the most stable training process. However, it is the least data-efficient. D-DDPG reaches the second-smallest gap to a reference solution with the highest data efficiency, but has the least training stability. DQN is the most stable model-free method but has the highest collision rate. FVI achieves a lower collision rate than DQN, but is less data-efficient and less stable. Based on these results, we provide suggestions for selecting a method according to whether a model is available, the budget for training, and whether stable training is required.