Active robot hand movements improve 3D object scanning accuracy
AURORA: Active Uncertainty-Driven Re-Orientation for In-Hand Reconstruction
RoboticsComputer Vision and Pattern Recognition
Summary
It’s hard for robot hands to see everything about an object they’re holding because parts get hidden. The authors created a system called AURORA that helps the robot decide how to move the object in its hand to see hidden parts better. It does this by figuring out which views are uncertain or new and then moving the object accordingly. This method leads to better 3D models of objects than just turning the object randomly or following preset motions.
What this means in practice
- •For robotics engineers: Enable robot hands to dynamically reposition grasped objects for improved 3D scanning and modeling during manipulation tasks.
- •For quality control teams: Use robot-assisted active scanning to capture detailed 3D models of small parts held in hands for enhanced visual inspection.
Authors
Feiyu Zhao, Yuetong Li, Chenxi Xiao
Abstract
Observing objects grasped by a robot hand is challenging due to severe visual occlusions. Although in-hand manipulation can expose hidden surfaces, existing approaches often rely on predefined or open-loop reorientation strategies that do not explicitly target under-observed regions. We propose AURORA, an active 3D reconstruction framework that closes the loop between online object-centric reconstruction and in-hand reorientation. At its core, Ray-GPIS estimates direction-wise reconstruction uncertainty along candidate viewing rays and selects next-best-view targets using an uncertainty--novelty objective, which are realized through an axis-conditioned in-hand rotation policy. The resulting RGB-D observations are fused incrementally using CAD-free 6D pose tracking and lightweight geometric reconstruction. Experiments demonstrate that AURORA improves reconstruction quality and information-acquisition efficiency over non-active rotation strategies, while Ray-GPIS also outperforms active view-planning baselines in reconstruction performance, action-ranking quality, and planning efficiency. Targeted ablations further validate its robustness to hand occlusion and pose errors. The project webpage is available at https://aurorahand.github.io/