Learning Reach-Avoid Task with Reinforcement Learning: Vectorized Simulation and Benchmark
RoboticsMachine Learning
Summary
The authors studied how well deep reinforcement learning (DRL) controls robotic arms in a task where the arm must reach a target while avoiding obstacles. They created a new, realistic test environment that is more complex than simple tabletop versions used before. Using advanced simulation tools, they tested DRL on different robot arms and got high success rates, but noticed that performance drops when moving from simple to realistic settings. Their work shows that more research is needed before DRL can fully solve these tasks in real-world conditions. They also shared their environment and code openly for others to use.
Authors
Jonas Weihing, Shahram Eivazi
Abstract
Deep reinforcement learning (DRL) has a longstanding tradition in addressing the reach-avoid task problem, especially for controlling robotic arms. While this task serves as a baseline environment within the research community, the ability of DRL to effectively learn the each-avoid task in complex and realistic scenarios beyond simplified and restricted tabletop settings remains uncertain. In this paper, we present, for the first time, a comprehensive benchmark for the reachavoid task that accurately captures real-world complexities without simplifications. We demonstrate a diverse range of settings for robotic arm reach-avoid task, which can be used for evaluating DRL research. We achieved this by utilizing the MuJoCo MJX physics engine and parallelizing both the simulation environment and DRL algorithms using the Brax library. We achieved state-of-the-art results with success rates of 96.1% (UR5e) and 98.8% (Franka Emika Robot) for the reach task and 86.8% (UR5e) and 95.2% (Franka) for the static reachavoid task. Our results indicate that while in previous works DRL agents could solve, for example, a reach task in a simplified setting perfectly, their agents performance collapses when evaluated in realistic scenarios. Overall, this work identifies that additional research is still required to claim the successful resolution of the robotic arm reach-avoid task using DRL. The environment and benchmarking code is available as open source at the following link