Closed-form triangle splatting speeds novel-view synthesis with mesh priors
CF-Splat: Closed-Form Triangle Splatting with Mesh Priors
Robotics
Summary
Rendering images of a 3D scene from new viewpoints usually takes a lot of computation and memory. The authors present CF-Splat, a method that uses a known 3D mesh and camera information to quickly calculate scene colors without slow iterative steps. Their approach solves the problem using math that finds the best fit in one go, making it much faster and more memory efficient than previous methods. This makes creating new views of scenes more practical on common laptops.
What this means in practice
- •For robotic perception teams: Generate high-quality new views from multi-camera data efficiently for robotics navigation and mapping using known mesh geometry.
- •For augmented reality developers: Produce realistic scene renderings quickly from known 3D models to enable responsive AR experiences on consumer devices.
Authors
Yixin Fang, Liang Zhao
Abstract
Splatting-based radiance-field representations have recently gained popularity in robotic perception, enabling high-quality novel-view synthesis (NVS) through explicit scene primitives. However, existing splatting methods typically rely on iterative nonlinear optimization that jointly optimizes millions of coupled primitive parameters, resulting in substantial computation time, peak GPU memory consumption, and susceptibility to local minima. In this paper, we propose CF-Splat, a novel closed-form triangle-splatting framework for efficient scene appearance estimation over a known mesh either pre-existing or provided by upstream mapping or reconstruction pipelines. Our key insight is that, by properly formulating the problem over the given mesh geometry and calibrated cameras, the rendered pixel colors can be expressed linearly with respect to the per-vertex spherical harmonics (SH) coefficients, allowing multi-view scene appearance estimation to be formulated as a global linear least-squares (LLS) problem without the need of iterative nonlinear optimization. To guarantee a well-posed solution under uneven multi-view observation coverage, we introduce adaptive SH regularization that constrains insufficiently supported high-order SH coefficients and yields a unique global minimizer. To ensure the scalability to large-scale scenes, we develop a triangle-structured matrix-free preconditioned conjugate-gradient solver that avoids explicitly constructing the large sparse normal system while producing the same LLS solution. Experiments on diverse real-world datasets demonstrate that CF-Splat achieves competitive or superior NVS quality compared with state-of-the-art splatting methods. While, CF-Splat requires approximately 27.5-378.0 times less computation time and 1.8 times lower peak GPU memory on average than 3DGS across scenes of different scales on a consumer-grade laptop.