CDIS: Cross-Dimensional Class-Agnostic 3D Instance Segmentation via 2D Mask Tracking and 3D-2D Projection Merging
2026-07-20 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial Intelligence
AI summaryⓘ
The authors developed a method called CDIS that helps robots identify and keep track of objects in 3D space without needing prior knowledge of the objects' types. Instead of just combining 2D images into 3D shapes which can cause confusion over time, their approach links tracked 2D object outlines with 3D segments to maintain clear and consistent object identities. This way, the method works well across time and space without special 3D training. Tests show it outperforms other similar methods in accuracy and consistency while being efficient for real-world use.
3D instance segmentationclass-agnostic segmentationzero-shot learning2D-3D associationsuperpointstemporal trackingrobotic perceptionobject identity managementspatial coherencecross-dimensional reasoning
Authors
Juno Kim, Hye-Jung Yoon, Yesol Park, Byoung-Tak Zhang
Abstract
Class-agnostic 3D instance segmentation is critical for robotic systems operating in unknown environments, enabling perception of previously unseen objects for reliable manipulation and navigation. Existing approaches typically project per-frame 2D instance masks into 3D and merge them, which often breaks object identities across time and yields fragmented 3D instances. We introduce Cross-Dimensional Class-Agnostic 3D Instance Segmentation (CDIS), a zero-shot framework that explicitly tracks 2D instance masks across frames and associates them with 3D superpoints, creating a feedback loop between 2D and 3D. This cross-dimensional reasoning links temporally stable 2D tracks with spatially coherent 3D regions, producing globally consistent 3D instance labels without any 3D-specific training. Experiments on benchmark datasets demonstrate that CDIS achieves higher accuracy and consistency than state-of-the-art zero-shot methods, while remaining efficient and scalable to diverse real-world environments.