FunArt decodes movable parts and motions from single 3D scans
FunArt: Decoding Functional Structure and Articulation from Generative 3D Latents
Computer Vision and Pattern RecognitionRobotics
Summary
Robots need to understand how parts on objects move to use them properly, but most methods require seeing those objects in motion. The researchers developed FunArt, a system that looks at a single 3D scan of an object to figure out its parts that can move, how they move, and how to interact with them. FunArt uses a special way to represent 3D objects and learns from general shapes to guess part boundaries and motion details without needing videos of the objects moving. This helps robots better plan how to manipulate objects just by seeing them once.
What this means in practice
- •For robot perception teams: Enable robots to identify movable parts and their motions from a single 3D scan without prior motion data to improve manipulation planning.
- •For 3d modeling software developers: Integrate automatic movable-part and articulation identification from static 3D data to streamline interactive object modeling workflows.
Authors
Dennis Rotondi, Abdelrhman Werby, Kai O. Arras
Abstract
To operate effectively in human environments, robots must identify articulated objects, segment their movable and interactive parts, and estimate their kinematic models. Existing articulated scene representations typically recover kinematics from observed interactions, while methods operating on static scans often decouple articulation from functional interactive elements. We present FunArt, a framework that constructs articulation-aware functional 3D scene graphs from posed RGB-D observations captured in a single static configuration. FunArt reconstructs object instances, converts their fused geometry directly into the O-Voxel representation of TRELLIS.2, and exploits its frozen, sparse-compression VAE as a structural prior. A lightweight query-based decoder combines compact object-level latents with dense, surface-aligned features to jointly segment movable parts and functional interactive elements while estimating motion type, axis, origin, and range. On the Articulate3D dataset, FunArt achieves state-of-the-art performance across movable-part segmentation, articulation estimation, and functional-element segmentation, both with and without ground-truth object input. In the end-to-end setting, it outperforms the strongest baselines by 1.5 AP_{50} points for movable parts, 2.8 AP_{50} points under joint origin-and-axis constraints, and 6.7 AP_{50} points for functional elements. These results demonstrate that generative 3D latents encode actionable structural cues that can initialize robotic perception and planning before physical interaction.