Robot plans multiple views to see hidden objects better
Receding-Horizon Next-Best-View Planning for Highly Occluded Target Perception
Robotics
Summary
Robots sometimes need to look at objects that are partly hidden by other things, so one picture isn’t enough. The authors created a method that helps a robot plan a few steps ahead to find better spots to look from, not just the best immediate spot. This planning helps the robot uncover more details about the hidden object than usual methods. They tested this on real and simulated robot arms and showed it works especially well for very hidden objects. When objects are not so hidden, simpler methods still work well and are faster.
What this means in practice
- •For industrial robot programmers: Plan sequences of camera viewpoints to better inspect objects obstructed by other parts or complex shapes, improving quality control in manufacturing.
- •For agricultural robot teams: Use multistep view planning to perceive and analyze plants with natural self-occlusion for improved monitoring and harvesting.
Authors
Ayse Gul Yildiz, Rekha Raja
Abstract
Robotic systems that must acquire information about a specific target often face severe occlusion from sur- rounding structures or from the target itself, such that a single camera viewpoint provides only partial information. Active perception must therefore select viewpoints that progressively reveal unobserved regions. However, conventional local next- best-view (NBV) methods typically select viewpoints according to their immediate utility, which can overlook views that provide limited information initially but enable more informative obser- vations from subsequent viewpoints. We formulated local target perception as a short-horizon sequential view-planning problem and proposed a Receding-Horizon Next-Best-View (RH-NBV) planner that evaluates candidate sequences of future viewpoints rather than isolated views. RH-NBV is evaluated on a UR5e manipulator with an eye-in-hand RGB-D camera in simulation as well as on physical robot against a gradient-based NBV planner, and random search, using a coffee mug and Stanford bunny under controlled occlusion and a tomato plant with natural self-occlusion. Under heavy occlusion, RH-NBV achieves substantially higher target information recovery, reaching 63.8% ROI coverage versus 34.2% and 85.5% occluded-surface recall versus 7.4% for the single-step NBV baselines on the most occluded mug configuration. A controlled horizon ablation shows that this improvement arises from multi-step reasoning rather than additional computation. RH-NBV provides the greatest benefit for strongly occluded targets, while myopic NBV remains competitive and substantially cheaper when visibility is less constrained.