Seg3DParts generates controllable 3D parts from single images
Seg3DParts: Segmentation-Grounded Controllable Part-Level 3D Generation
Computer Vision and Pattern Recognition
Summary
Creating detailed 3D objects made of separate parts from just one image is hard because parts can hide behind each other and boundaries are unclear. The authors developed Seg3DParts, a system that uses image segments to directly guide the 3D creation of each part, making sure they fit together well. They also built a large dataset called PartObjectNet to help learn these parts. This system can produce high-quality 3D models with clear parts that can be controlled, useful for editing or combining objects.
What this means in practice
- •For 3d modeling teams: Create 3D models with controllable parts directly from images for use in editing and reassembly tasks.
- •For augmented reality developers: Generate detailed, editable 3D assets from photographs to improve interactive AR experiences.
Authors
Jiantao Lin, Meixi Chen, Yingjie Xu, Chenbo Fu, Leyi Wu, Hao Chen, Yinchuan Li, Ying-Cong Chen
Abstract
Part-level 3D assets are essential for editing, reassembly, and interaction, yet recovering such structure from a single image remains challenging due to occlusion, ambiguous boundaries, and the need for coherent multi-part reasoning. Existing approaches struggle to achieve both controllable part-level generation and coherent multi-part structure, as part identity and spatial allocation are typically inferred implicitly. We present Seg3DParts, a segmentation-grounded framework for controllable part-level 3D generation from a single image. By treating segmentation as an explicit grounding signal, our method defines part identity during generation, enabling each component to be anchored to a corresponding image region. To ensure coherent assemblies, we introduce structured cross-part interaction that allows components to exchange global context throughout the generative process. As a result, Seg3DParts directly generates well-aligned part meshes in a shared canonical space without post-hoc alignment, supporting flexible and controllable decomposition. We further introduce PartObjectNet, a large-scale dataset with over 200K objects and 1M annotated parts. Experiments demonstrate that Seg3DParts achieves superior geometry quality, cross-part coherence, and part-level controllability over existing methods.