AI summaryⓘ
The authors created a new way to turn face sketches into 3D models by using three main tools: neural networks, a 3D face model called Valley Girl, and a technique called Active Snake Contours. They trained neural networks to recognize facial expressions in sketches by detecting facial muscle movements. Then, they applied these expressions to the 3D face model. Finally, they used Active Snake Contours to adjust the 3D model so it matches the sketch more closely.
3D face modelingface sketchesConvolutional Neural NetworksFACS Action Unitsfacial expressionsparametric 3D face modelValley Girl modelActive Snake Contours
Abstract
Generating 3D models from face sketches is an active topic of research in Computer Graphics due to its potential to tremendously facilitate the modeling of faces for both professional 3D arists and novices. Motivated by the observation that facial expressions are responsible for significantly altering and shaping the contours in our faces, we combine both expression detection and 3D model generation in our approach. The result is a novel approach to generating 3D models from sketches which relies on three components: Convolutional Neural Networks, a parametric 3D face model (Valley Girl), and Active Snake Contours. For the first time in the literature, CNNs are trained (using our own generated dataset) to detect the expression in the given sketch through detecting the active FACS Action Units. The expression is then duplicated on Valley Girl to obtain a 3D model with a similar expression. Active Snake Contours are then used to find the transforms needed to close the gaps between that model and the given sketch.