3D Human Face Reconstruction with 3DMM face model from RGB image
2026-05-05 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionGraphics
AI summaryⓘ
The authors developed a method to create a 3D model of a human face from a single 2D photo using convolutional neural networks. They explained that while CNNs are good at image tasks, training them usually needs lots of labeled data, which is hard to get for detailed face shapes. To help, the authors used a pipeline combining face and landmark detection, estimating 3D face model parameters, and a rendering step to generate the 3D face. Their approach addresses challenges in creating realistic facial details that earlier models struggled with.
convolutional neural networks3D face reconstruction2D face imagesface landmarks3D morphable modelface detectionregressionsoft renderingphoto-realistic synthesisimage processing
Authors
Zhangnan Jiang, Zichen Yang
Abstract
Nowadays as convolution neural networks demonstrate its powerful problem-solving ability in the area of image processing, efforts have been made to reconstruct detailed face shapes from 2D face images or videos. However, to make the full use of CNN, a large number of labeled data is required to train the network. Coarse morphable face model has been used to synthesize labeled data. However, it is hard for coarse morphable face models to generate photo-realistic data with detail such as wrinkles. In this project, we present a pipeline that reconstructs a human face 3D model from a single RGB image. The pipeline includes face detection, landmark detection, regression of 3DMM model parameters, and soft rendering. Mentor: Zhipeng Fan (Email: zf606@nyu.edu) Code Repository: https://github.com/SeVEnMY/3d-face- reconstruction Code Reference: https://github.com/sicxu/Deep3DFaceRecon pytorch