PartMat: Material-Aware 3D Part Decomposition with a Single Global Latent
2026-08-03 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial IntelligenceGraphics
AI summaryⓘ
The authors developed PartMat, a method for breaking down 3D objects into parts based on materials like fabric or metal, instead of just how the parts function. Unlike previous methods that treat each part separately and get slower with more parts, their approach uses one combined representation to handle all parts at once, making it faster. They also use machine learning techniques including a diffusion model and reinforcement learning to improve how accurately parts are assigned and separated. Additionally, they added a step to refine detailed shapes in each part. Their experiments show PartMat is better at recognizing material-based parts and keeps good shape quality efficiently.
3D part decompositionmaterial-aware segmentationlatent representationvariational autoencoder (VAE)diffusion modelreinforcement learningsparse-voxel flow matchinggeometry post-processing
Authors
Guangming Fu, Jin Song, Yiyun Fei, Guoqiu Li, Ruigao Yang, Jianan Jiang
Abstract
Part-level 3D generation has recently attracted increasing attention for producing structured and editable 3D assets. However, existing methods typically decompose objects according to functional semantics rather than the editable material boundaries (e.g., fabric, wood, metal) required in practical 3D applications such as interior design. Additionally, current methods often generate parts independently, causing computational costs to scale linearly with the part count. To address these limitations, we present PartMat, an efficient material-aware 3D part decomposition pipeline that represents multi-part geometry with a single global latent. Given a reference image and a single whole-object geometry, PartMat decomposes the object into parts that follow material boundaries. First, we propose PartVAE to learn such a unified representation and decode all material parts in a single forward pass, thereby decoupling inference cost from the number of parts. Second, with this representation, a diffusion model is trained for part generation and refined via reinforcement learning for accurate material assignment and overlap suppression. Finally, to recover fine-grained geometric details, we introduce a sparse-voxel flow-matching model with part attention for geometry post-processing. Extensive experiments demonstrate that PartMat significantly outperforms existing baselines in material-aware decomposition accuracy and achieves comparable geometric quality, while maintaining efficient inference.