Improving 3D pose detection of shiny and smooth objects for robots
Generalizable 6D Pose Estimation of Textureless Objects with Planar-based Gaussian Splatting
Robotics
Summary
Figuring out how shiny or smooth objects are positioned in 3D space is tricky because they don’t have clear patterns to recognize. The authors created a method called PG-Pose that uses special 3D shapes called Gaussian splats combined with planar surfaces to better understand an object’s geometry from multiple pictures. This helps a computer match new images to the 3D model and fine-tune the object’s pose with higher accuracy. They tested their method on hard-to-see objects and found it works better than previous approaches, even helping a robot successfully pick up an object it had never seen before.
6D pose estimationtextureless objectsplanar-based Gaussian splatting3D reconstructionRGB images2D-3D correspondencepose optimizationpoint cloudsindustrial roboticsobject grasping
Authors
Jie Lu, Hengtan Zhang, Li Gong, Pengpeng Wang, Xianjia Yu, Jinxiang Deng, Tomi Westerlund, Zhongxue Gan, Lirong Zheng, Zhuo Zou
Abstract
Estimating the 6D pose of textureless objects without prior CAD models remains a critical challenge due to the lack of appearance features. While recent generalizable approaches alleviate the dependence on object-specific models, their performance on low-texture objects is often limited by insufficient geometric constraints in the underlying representations. In this work, we propose PG-Pose, a geometry-aware framework combining Planar-based Gaussian Splatting (PGS) reconstruction and Geometry-driven pose optimization. In the offline representation extraction stage, three distinct representations of the object are extracted from multi-view reference RGB images with known poses. PG-Pose reconstructs a 3D Gaussian representation and renders high-fidelity depth maps to generate 3D point clouds through back projection. In the online pose inference stage, the initial pose of the input image is estimated by 2D-3D correspondence matching between the input image and the reconstructed 3D point clouds, followed by a PGS-Refiner for iterative pose optimization. Evaluations on the OnePose-LowTexture datasets, PG-Pose achieves an average accuracy of 94.2% ADD(S)@0.1d, with a 2.1% improvement average accuracy compared with the state-of-the-art (SOTA) GS-based approach. To further demonstrate the effectiveness of PG-Pose for industrial robots in grasping tasks, we deploy it on a dual-arm industrial robot and successfully realize the grasping task on an unseen object.