Real-time obstacle-aware reachability maps speed robotic arm movement
HERON: Real-Time Obstacle-Aware Reachability Maps for Mobile Manipulators
Robotics
Summary
Mobile robots with arms find it hard to quickly decide how and where they can move their arm around obstacles. The authors developed HERON, a fast method that uses a type of artificial intelligence to predict safe and reachable areas for the robot’s arm in real time. HERON learns from detailed simulations and actual camera data, running efficiently on small onboard computers. This lets robots update their movement plans about ten times every second, which is much faster than previous methods that took an hour or more. HERON is mostly accurate, except near the edges of the robot’s reach.
What this means in practice
- •For robotic system integrators: Enable mobile manipulators to quickly update arm reachability maps with obstacle awareness for safer and more flexible object handling in cluttered environments.
- •For industrial robot operators: Improve runtime efficiency and responsiveness of robots used in warehouses or factories by rapidly computing precise reachability around obstacles.
Authors
G. Dupoiron, F. Michaud, J. P. Roberge
Abstract
Mobile manipulator robots rarely reposition their base during manipulation, because doing so requires real-time knowledge of how well the arm can reach its surroundings among obstacles. Reachability maps (RMs) encode this knowledge, but existing methods are slow, coarse (typically 5 cm), or ignore real obstacles. This article asks whether a neural network can predict an obstacle-aware RM at 2 cm resolution from a live depth camera, on embedded hardware, fast enough for online use. We present HERON, a 3D U-Net trained on RMs computed with the GPU-accelerated inverse kinematics of cuRobo, first on synthetic scenes and then on real depth data, and deployed on a Jetson AGX Orin with TensorRT and INT8 quantization. HERON predicts the RM of a 2.56 m cube around the arm in 115 ms on average, for a measured system rate of 8.57 Hz, close to the 10 Hz update rate of the obstacle map, whereas computing the same map with cuRobo takes about 90 min. Against the cuRobo ground truth, 95% of the voxels lie within 1 class out of 51 with the FP32 model and within 3 classes with the INT8 model, with a Pearson correlation of 0.99 and a quadratic weighted kappa above 0.9. Accuracy degrades mainly at the workspace boundaries.