RealDenseFace: Real-time Monocular 3D Face Reconstruction from Dense UV-space Priors
2026-08-10 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors created RealDenseFace, a fast method to build 3D face models from just one photo. They use a special math approach to quickly adjust a 3D face shape so it matches details predicted by a neural network. Their method works in two steps: first, the network guesses how the face's surface matches the image and depth; second, a solver fine-tunes the 3D face to fit these guesses. This approach is much quicker than older methods and still very accurate, even working in real-time for face tracking.
3D Morphable Model (3DMM)monocular 3D face reconstructionnonlinear least-squares optimizationGauss-Newton solverUV-space mapscorrespondence maprelative-depth mapreal-time trackingoffline sequence reconstructionneural networks
Authors
Linzhou Li, Tianjia Shao, Kun Zhou
Abstract
Recent monocular 3D face reconstruction methods achieve high fidelity by fitting a 3D Morphable Model (3DMM) to dense priors predicted by networks, but the optimization stage is computationally expensive, often taking tens of seconds per image. We present RealDenseFace, a real-time optimization-based 3D face reconstruction method with dense UV-space network predictions. Our key idea is to formulate 3DMM fitting as a nonlinear least-squares problem and solve it with a tailored Gauss-Newton solver that converges in only a few iterations. The reconstruction is conducted in two stages. In the first stage, the network predicts two dense UV-space maps from a single RGB image: a correspondence map for UV-to-image alignment, and a relative-depth map for geometric constraints along the viewing direction. In the second stage, the solver fits per-vertex targets sampled from these maps at the vertex UV coordinates. The solver supports all three reconstruction settings: single-image fitting, offline sequence reconstruction, and online tracking. Our method achieves state-of-the-art accuracy on the NeRSemble SVFR benchmark. The online tracker runs at 80+ FPS, and the offline sequence reconstruction is over 20 times faster than previous optimization-based baselines.