Summary
The problem addressed is figuring out the exact positions and orientations of several identical objects in a single photo, without knowing anything about the objects beforehand. The authors introduce a method called PROSE that matches parts of these objects based on visual features without needing any prior information or training. PROSE improves these initial matches by checking consistency among groups of objects to find reliable connections. Using these consistent matches, it calculates how the objects are oriented relative to each other. They also created a test set called PRENCH to measure how well this method works, showing PROSE does better than previous approaches that try to adapt single-object methods to this multi-object scenario.
6D pose estimationobject instancesrelative posecorrespondencescycle consistencymultimodal featuresprior-free methodsbenchmark datasetBOP datasetspose refinement
Authors
Behdad Khodabandehloo, Andrea Caraffa, Davide Boscaini, Fabio Poiesi
Abstract
Object 6D pose estimation formulations have progressively reduced reliance on object-specific priors, evolving from explicit 3D models to multi-view object captures to single reference images. We take this progression to its extreme by introducing prior-free relative 6D pose estimation, which lifts the assumption of knowing which object is to be posed within the scene. This novel setting aims to estimate the relative poses of multiple instances of an unknown object within the same image, without requiring CAD models, templates, or reference images. We solve this by formulating a novel method (PROSE) that finds coarse correspondences between object instances using multimodal foundation features, thus requiring no training. We refine these correspondences by imposing cycle consistency across tuples of instances, and leverage the resulting globally consistent correspondences to estimate the relative 6D pose between any pair of instances. To enable systematic evaluation, we design a novel benchmark (PRENCH) built from three multi-instance BOP datasets and enriched with task-specific metadata. PROSE consistently outperforms baselines obtained by adapting state-of-the-art single-image methods to the proposed setting, while requiring neither task-specific supervision nor additional learned components. Project website: https://tev-fbk.github.io/PROSE/