Vision tomography reconstructs 3d face and skull shapes from images
Vision Tomography: Towards Portable 3D Bidirectional Face-Skull Inference from Visual Data
Computer Vision and Pattern Recognition
Summary
Getting detailed 3D images of a person's face and skull usually requires expensive machines and exposes people to radiation. This paper presents a new method called Vision Tomography that can create 3D models of a face and skull just from pictures or sketches, without special equipment or safety risks. The authors built a unique dataset and a model that links faces and skulls, allowing them to predict one from the other and even simulate changes in appearance. Their approach shows promise for uses in medicine, forensics, and history without relying on traditional CT scans.
What this means in practice
- •For forensic investigators: Create accurate 3D face and skull models from photos or sketches to help identify unknown persons without CT scanning.
- •For medical imaging teams: Predict surgical outcomes by editing paired 3D face and skull shapes reconstructed from non-radiative visual data.
Authors
Zidu Wang, Meng Xu, Miao Xu, Hengyuan Ma, Jiankuo Zhao, Xutao Li, Zhen Lei, Xiangyu Zhu
Abstract
Traditional computed tomography (CT) can capture paired 3D face and skull geometry, but it remains costly, fixed in location, and exposes patients to radiation. For applications such as remote diagnostics, digital healthcare, forensic identification, missing-person investigation, and archaeological analysis, there is an urgent need for a portable, safe, and accessible solution to capture 3D skull geometry. In this work, we introduce Vision Tomography (VT), a concept and framework capable of reconstructing paired 3D face-skull geometry from various visual modalities, and also enabling joint manipulation of 3D facial and skull geometry. To achieve this, we first construct a dataset of over 200 samples, where each case includes a CT-based skull, a CT-based face, and a high-fidelity textured face scan. Secondly, we propose a novel dense ray matching registration method that ensures topological consistency across face, skull, and their tissue correspondences. Building on this registration, we introduce the 3D Bidirectional Face-Skull Morphable Model (BFSM) as the foundation of Vision Tomography, which enables shape inference between the face and skull through a shared coefficient space, while also modeling tissue thickness variation to support one-to-many facial reconstructions from the same skull, reflecting individual facial changes such as age or body fat. Finally, we demonstrate the effectiveness of Vision Tomography across various forensic and medical applications, including paired 3D face-skull reconstruction from images or sketches and predicting surgical outcomes with joint face-skull editing. Extensive experiments confirm the robustness and accuracy of our method. Vision Tomography is available at https://github.com/wang-zidu/BFSM