OptimusMesh generates compact 3D triangle meshes from point clouds
OptimusMesh: Compact Autoregressive Mesh Generation from Point Clouds via Sparse Latent Pivots
Computer Vision and Pattern Recognition
Summary
Creating 3D shapes from messy point clouds is tricky because point clouds are just scattered dots without order, while 3D shapes (meshes) have connected triangles and complex structures. The authors developed OptimusMesh, which compresses a large point cloud into a small set of key points called latent pivots. This small set helps generate the shape’s vertices and faces step-by-step, producing detailed but much simpler 3D shapes. Compared to other methods, OptimusMesh uses fewer points to guide the process and outputs fewer triangles without losing shape quality.
What this means in practice
- •For 3d modeling software developers: Generate compact and detailed 3D meshes directly from point cloud scans, reducing mesh size and post-processing time.
- •For augmented reality app makers: Create simpler 3D mesh representations from scanned environments for efficient rendering on mobile devices.
Authors
Mazhar Iqbal, Naoya Chiba, Xuanmeng Sha, Tomohiro Mashita, Yuki Uranishi
Abstract
Generating compact and geometrically faithful 3D meshes directly from point clouds remains a fundamental challenge. Point clouds are unordered and sparse, whereas meshes exhibit irregular structure and varying topology. As a result, many existing approaches rely on implicit representations followed by surface extraction or reconstruction. Although effective, these pipelines can produce dense or over-smoothed meshes, often requiring computationally expensive post-processing and simplification. We present OptimusMesh, a framework for direct compact triangle mesh generation from point clouds using sparse latent pivot conditioning. Our key idea is to compress $2{,}048$ oriented input points into only $16$ sparse latent pivots, reducing the geometric conditioning set by $128\times$. These pivots provide a compact structural representation shared across a two-stage autoregressive framework that first generates mesh vertices and then predicts triangular faces conditioned on the generated vertices and the same pivots. Compared with the evaluated recent point-cloud-conditioned autoregressive methods, which use $257$ decoder-conditioning tokens, OptimusMesh uses only $16$, yielding a $16.1\times$ shorter conditioning sequence. Experiments show that OptimusMesh produces the most compact outputs among the compared recent autoregressive methods, using $25.7\%$--$94.1\%$ fewer faces while maintaining competitive geometric fidelity and distributional quality.