Mesh extraction method delivers watertight object shapes from 3D scenes
MEGA: Object-Level Mesh Extraction from 3D Gaussian Splatting via Spatial Visual Distillation
Computer Vision and Pattern Recognition
Summary
Extracting 3D shapes from complex scenes is tricky, especially when breaking them down into individual objects with clear boundaries. The authors propose MEGA, a new method that first separates objects in a 3D scene made from fuzzy blobs (called Gaussian splats) and then builds precise, watertight 3D meshes for each object. They use a process called Spatial Visual Distillation, where multiple views of each object guide the mesh reconstruction, making the shapes more accurate. This helps create detailed 3D objects that can be used for both visuals and physical simulations.
What this means in practice
- •For 3d graphics developers: Create detailed, accurate 3D object meshes for realistic rendering and interaction in virtual environments using MEGA’s object-level extraction.
- •For robotics engineers: Generate precise object meshes for robot perception and manipulation in complex scenes where object boundaries must be clear.
Authors
Liwei Liao, Yingkui Zhang, Qianqian Tong, Ronggang Wang
Abstract
Mesh extraction from 3D Gaussian Splatting (3DGS) aims to endow 3D Gaussians with accurate geometric structures, enabling explicit and precise 3D occupancy. However, existing methods primarily focus on scene-level mesh extraction, making them unable to represent object-level occupancy and often resulting in non-watertight surfaces. To overcome these limitations, we propose \textbf{MEGA} (\underline{M}esh \underline{E}xtraction from \underline{GA}ussians), a ``segment-then-mesh'' framework for extracting object-level, watertight meshes from complex 3DGS scenes. At the core of MEGA are \textbf{Spatial Visual Distillation (SVD)} and a mask-guided neural surface reconstruction module. SVD treats the 3DGS model as a teacher, sampling diverse camera poses and rendering the corresponding views of each segmented object. These observations are then used to train a mesh reconstruction model through photometric supervision. Extensive experiments on several widely used benchmarks demonstrate that MEGA achieves state-of-the-art performance in recovering accurate object-level 3D occupancy. Moreover, MEGA enables complex physical interactions by combining high-quality object-level meshes for geometric occupancy with 3DGS representations for photorealistic rendering.