Neural potential fields plan robot arm paths directly from images
SE(3) Neural Potential Fields for 6-DoF Trajectory Planning Directly from Images Without Explicit 3D Reconstruction
RoboticsArtificial Intelligence
Summary
Planning how a robot arm moves to grab something without bumping into things is tricky and usually involves making a 3D map first. The authors developed a method that plans safe robot arm paths directly from images, without making explicit 3D maps. Their approach uses learned fields that guide the arm around obstacles and reliably get close to the target grasp. Tests on a real robot show it works faster and safer than traditional methods relying on 3D reconstructions.
What this means in practice
- •For robotic arm developers: Plan collision-free 6-DoF arm trajectories directly from images without costly 3D reconstructions.
- •For warehouse automation teams: Deploy faster obstacle-aware grasp planning for pick-and-place robots using image-based potential fields.
Authors
Jeffrey Eiyike, Masoud Ataei, Elvis Gyaase, Vikas Dhiman
Abstract
Reaching a 6-DoF grasp pose in clutter requires a collision-free trajectory, conventionally obtained by reconstructing the scene in 3D and planning inside that reconstruction, at the cost of its accuracy and compute. Potential fields learned directly from images remove that dependency but inherit the classical weakness of artificial potential fields: where attractive and repulsive gradients cancel, the descent grazes the obstacle instead of going around it, and can stall short of the goal. We present an SE(3) neural potential field learned from posed RGB images and supervised with a navigation function, the geodesic distance to the grasp through free space recovered from those same images during training, which removes both failures. On two tabletop scenes, from obstacle-blocked starts executed on a UR10, the field converges within 3 cm of the grasp from every start and every path it executes is collision-free against the ground-truth geometry, against 25% and 0% under image supervision alone; mean clearance rises from under a centimeter to 8.6-8.8 cm and arm-link contacts fall from 20.6-50.4% to 2.7-5.5% of executed configurations. Executed grasp success is 90.0% and 40.0% on the two scenes, the residual failures being refusals of the Cartesian executor rather than of the field. Planning takes about 2 s against 67-133 s for RRT* on a reconstruction of the same images, though under a common offline harness the two are comparable: the deployed margin is the cost of collision-checking a dense reconstruction, not planner complexity.