CNS-Edit++: Category-Agnostic 3D Editing with Coupled Neural Shape Representation
Computer Vision and Pattern RecognitionGraphics
Summary
The authors developed a new method called CNS-Edit++ for editing 3D shapes using neural networks. It builds on an earlier method by combining a global code that understands the overall shape with a detailed 3D feature map for local changes. Their method works for specific object types and more general 3D models, allowing edits like resizing, copying, or dragging parts of a shape. They also introduced techniques to keep unchanged parts intact during editing. Tests showed their method works better than existing approaches.
Authors
Jingyu Hu, Weilong Yan, Zhengzhe Liu, Haipeng Li, Ka-Hei Hui, Hao, Zhang, Chi-Wing Fu
Abstract
This paper presents a latent-space 3D shape editing framework built upon a coupled neural shape (CNS) representation and a neural feature volume optimization. This work extends CNS-Edit, built on Coupled Neural Shape optimization, to CNS-Edit++, by generalizing the category-specific coupled representation to category-agnostic 3D shape editing with foundation models. The Coupled Neural Shape (CNS) representation couples a global latent code that captures high-level shape semantics with a 3D neural feature volume that provides spatial context for local shape manipulation. Then we formulate a coupled neural shape optimization procedure that co-optimizes these two components subject to a given editing operation. Our framework can be instantiated on both the category-specific 3D inversion model and category-agnostic 3D foundation models. We provide various shape editing operators, including copy, resize, delete, mix, point-wise drag, and region-wise drag, each of which is formulated as an objective to guide the CNS optimization. To preserve regions outside the editing area, we further introduce two complementary region-wise control mechanisms, i.e., KV-cache replacement and latent feature regularization. Extensive quantitative and qualitative evaluations across different 3D generative models demonstrate the strong capabilities of our approach over state-of-the-art solutions.