PIXIE: A Zero-Shot texture-invariant 6D pose estimation framework for unseen objects with assembly defects
Computer Vision and Pattern RecognitionRobotics
Summary
The authors introduce PIXIE, a new method to figure out where an object is and how it's rotated in a 3D space using just a regular photo and a simple 3D model without colors or textures. Instead of relying on detailed textures or heavy data processing, their approach uses geometric shapes and features from synthetic depth images to match the real photo. This makes PIXIE work well even when objects have damage, missing textures, or different lighting. They tested it on common benchmarks and made a new dataset to show it works in real-world messy scenarios.
Authors
Leon Jungemeyer, Alejandro Magaña, Gautham Mohan, Matthias Karl, Daniel Werdehausen
Abstract
6D pose estimation remains a key challenge in robotics and computer vision, particularly in industrial environments. The deployment of currently available data-driven methods is often limited by resource-intensive data pipelines, reliance on textured 3D models, and sensitivity to geometric deviations caused by damages or assembly defects. We present PIXIE, a zero-shot framework that estimates the 6D pose of an object from an RGB image using only an untextured 3D model. Synthetic depth and normal maps are rendered from sampled reference viewpoints and matched to the query image via a pretrained cross-modality feature matcher. Matched keypoints are back-projected to obtain 2D--3D correspondences for PnP-based pose estimation. Relying exclusively on geometry makes the method inherently robust to lighting and texture variation, while correspondence filtering handles geometric deviations between the model and physical object. We evaluate on widely-used public benchmarks, reporting state-of-the-art results on texture-less objects without object-specific training, and introduce a novel dataset with assembly defects, texture variations, and occlusion to demonstrate real-world applicability.