MeshCarve speeds up artisan mesh creation using compact latent spaces
MeshCarve: Artisan Mesh Generation with Flow Matching in Compact Latent Spaces
Computer Vision and Pattern Recognition
Summary
Making detailed 3D mesh shapes by hand can be slow with traditional computer methods. The authors created MeshCarve, a faster way that breaks the problem into parts—first figuring out where the points in the shape go, then how they connect, instead of doing both at once. They use smart data shortcuts called VAEs to shrink the information needed without losing detail. This method works better and quicker than older ways, helping software create complex 3D shapes more efficiently.
What this means in practice
- •For 3d modelers: Generate detailed artisan-quality meshes faster by using a new method that compresses and separates vertex and edge data.
- •For game developers: Create complex 3D assets with improved speed and fidelity by applying MeshCarve’s compact latent space generation approach.
Authors
Xiyu Wang, Ruocheng Wu, Yufei Wang, Zhihao Li, Lanqing Guo, Bihan Wen
Abstract
Prior artisan mesh generation works largely predict face tokens autoregressively, which makes inference slow. Recent methods instead flow match continuous latents built by Variational AutoEncoders (VAEs), but reconstruction quality drops significantly when geometry and topology are jointly encoded, and further when the latent space is compressed. We present MeshCarve, a flow matching method that generates entirely in compact latent spaces, generating vertex positions and edge connections separately and sidestepping the difficulty of a joint compact latent. To shorten the token sequence, we propose a hierarchical sparse transformer backbone, instantiated as VertexVAE and EdgeVAE. Instead of encoding fields over the surface voxels, both VAEs anchor on discrete vertices in their latent spaces, which drastically reduces the token sequence length, and our spatial-aware compression shortens it further without costing reconstruction. VertexVAE directly encodes vertex occupancy. For connectivity, we propose vertex-link encoding, which turns arbitrary connectivity between vertices into fixed-length continuous per-vertex embeddings and recovers complex artistic topology faithfully. MeshCarve combines these VAEs with an anchor generator and flow matches on the shortened token sequences. It shows advantages over state-of-the-art autoregressive and flow matching methods on Objaverse and generalizes to Toys4K. To the best of our knowledge, it is among the first artisan mesh generation methods whose every generative stage runs in a spatially compressed latent, with a token sequence only a fraction of the most compressed previous autoregressive and flow matching works.