Separable embeddings overcome pose errors in 3D shape learning

SIEVE: Overcoming Topological Obstruction in Equivariant Self-Supervised Learning

Machine LearningArtificial IntelligenceComputer Vision and Pattern Recognition

Summary

Learning 3D shapes from partial views is tricky because usual methods confuse an object's orientation without a stable frame of reference. The authors explain a mathematical reason why typical masked autoencoder techniques fail when applied to networks that understand 3D rotations. They propose a new method called SIEVE that looks at shapes in a way that keeps orientation information intact while hiding fine details. This lets computers make better sense of 3D objects, improving performance on recognizing new shapes.

What this means in practice

  • For 3d vision engineers: Improve 3D object recognition models by using invariant scalar embeddings to handle orientation issues in training data.
  • For robotics software developers: Enhance robot perception systems with representations that generalize better to new object shapes and orientations.

Authors

Jongann Lee, Sun Woo Park, Yun Young Choi

Abstract

We identify a theoretical incompatibility between 3D point cloud masked autoencoding and SE(3)-equivariant networks. We prove that standard masking triggers a topological collapse, effectively forcing the model to hallucinate orientation without a reference frame. To resolve this, we introduce SIEVE (Scalar Invariant Extraction Via Equivariance), an SE(3)-equivariant self-supervised framework that compresses point cloud geometry into invariant scalar embeddings. At its core is the spherical projection mask, which preserves global pose while obfuscating local semantics, sidestepping the obstruction by construction. Experiments confirm that SIEVE circumvents the predicted collapse where constant and noise masks fail. On downstream point cloud tasks, the resulting embeddings improve over coordinate-based baselines and hand-crafted geometric descriptors, with the largest gains on generalization to unseen shape categories.
View PDFOpen arXiv