Pose estimation improves for textureless objects using planar gaussians
Generalizable 6D Pose Estimation of Textureless Objects with Planar-based Gaussian Splatting
Robotics
Summary
Estimating the position and orientation of objects that have no texture is hard because usual image features don't work well. The authors introduce PG-Pose, which builds a detailed 3D model from multiple images and uses geometric clues to guess the object's pose more accurately. Their method improves performance on challenging objects compared to previous ones and was tested successfully on a robot that picks up unseen items.
What this means in practice
- •For industrial robot operators: Enable robots to find and grasp unseen textureless objects accurately using improved pose estimation.
- •For warehouse automation engineers: Improve object handling by estimating poses of untextured items without prior models from multi-view RGB images.
Authors
Jie Lu, Hengtan Zhang, Li Gong, Pengpeng Wang, Xianjia Yu, Jinxiang Deng, Tomi Westerlund, Zhongxue Gan, Lirong Zheng, Zhuo Zou
Abstract
Estimating the 6D pose of textureless objects without prior CAD models remains a critical challenge due to the lack of appearance features. While recent generalizable approaches alleviate the dependence on object-specific models, their performance on low-texture objects is often limited by insufficient geometric constraints in the underlying representations. In this work, we propose PG-Pose, a geometry-aware framework combining Planar-based Gaussian Splatting (PGS) reconstruction and Geometry-driven pose optimization. In the offline representation extraction stage, three distinct representations of the object are extracted from multi-view reference RGB images with known poses. PG-Pose reconstructs a 3D Gaussian representation and renders high-fidelity depth maps to generate 3D point clouds through back projection. In the online pose inference stage, the initial pose of the input image is estimated by 2D-3D correspondence matching between the input image and the reconstructed 3D point clouds, followed by a PGS-Refiner for iterative pose optimization. Evaluations on the OnePose-LowTexture datasets, PG-Pose achieves an average accuracy of 94.2% ADD(S)@0.1d, with a 2.1% improvement average accuracy compared with the state-of-the-art (SOTA) GS-based approach. To further demonstrate the effectiveness of PG-Pose for industrial robots in grasping tasks, we deploy it on a dual-arm industrial robot and successfully realize the grasping task on an unseen object.