KaiNinja extends 3D image generators to create separate parts quickly
Kaininja: Extending Native 3D Generators to the Part Level
GraphicsArtificial IntelligenceComputer Vision and Pattern Recognition
Summary
Turning a single image into a full 3D model is common, but those models are usually one piece. The authors found a new way to make 3D models where each part is separate and can be edited. Their method uses two volumes instead of one to solve a problem with overlapping parts. This approach works faster and better than previous methods without needing extra segmentation steps.
What this means in practice
- •For game developers: Create detailed 3D game assets with editable individual parts directly from images without extra segmentation steps.$Commercial implications: Enables production of high-quality part-level 3D models for games faster and more accurately, improving asset pipelines and customization.
- •For animation studios: Generate 3D characters and objects from images where parts can be independently rigged and animated without post-processing.
Authors
Ruihan Yu, Lian Fu, Muyao Niu, Zheng-hui Huang, Yu-Ju Tsai, Sho Kuno, Fengbo Lan, Yonghao Yu, Erwin Wu, Ming-Hsuan Yang, Kaipeng Zhang, Zhixiang Wang
Abstract
Native 3D generators turn one image into a single mesh. TRELLIS.2 and its peers deliver high-fidelity non-watertight geometry with materials, but the output is one fused object, while downstream work such as editing, rigging and simulation operates on part-level assets. A naive idea is to run a 3D segmentation network on the fused mesh that TRELLIS.2 generates, but such pipelines are slow and bounded by the accuracy of the segmentation. We want a simple way to extend an existing native 3D generator to the part level. But we face a critical problem: the O-Voxel grid stores one sheet of surface per voxel, so a single volume cannot represent the interface where two parts touch, at any resolution. We introduce a dual-volume representation to solve this problem and put forward KaiNinja, a part-level extension of TRELLIS.2 built on a dual-volume form of its O-Voxel representation. KaiNinja keeps the generation speed and quality of TRELLIS.2 while extending it to the part level, with no mask or segmenter in the pipeline. Its training data come from sources of many kinds, including CAD models and assets authored by an LLM-driven agent; to our knowledge it is the first 3D generative model trained on agent-authored part data. Surprisingly, we also find that whole-object fidelity improves over the same backbone fine-tuned on the same dataset. Against part generation pipelines of different paradigms, it lowers whole-object Chamfer distance by 40% and raises strict part F-score by 16%.