Joint 2D-3D model speeds up and improves femur shape reconstruction from X-rays
A Joint 2D-3D Statistical Shape Model for Orthopedic Reconstruction
Computer Vision and Pattern Recognition
Summary
Reconstructing a 3D shape of the thigh bone from 2D X-ray images is tricky because X-rays only show flat images without depth. The authors created a new model that learns the connection between 2D X-ray shapes and 3D bone shapes together, instead of separately. This allows them to quickly guess the 3D shape from new X-rays without slow step-by-step matching. Their method works better and about four times faster than older models, making it more useful for planning surgeries and checking implants.
3D reconstruction2D X-raysstatistical shape modelprincipal component analysislatent spacefemurorthopedic surgeryimage segmentation
Authors
Florence Dell'Aniello Picard, Pranav Poudel, Nairouz Shehata, Frédéric Lavoie, Herve Lombaert
Abstract
Three-dimensional femoral reconstruction from radiographs supports surgical planning, implant sizing, and post-operative follow-up, but remains ill-posed as X-ray projections discard depth information. Existing methods often incorporate a 3D statistical shape model (SSM) as a shape prior to guide reconstructions toward anatomically plausible shapes, relying on iterative 3D-to-2D projection matching. Yet, these approaches are computationally expensive and constrain their SSM to a single dimensionality, leaving the statistical relationship between 2D observations and 3D geometry largely unexploited and unexplored. We instead propose a joint 2D-3D SSM that explicitly captures the co-variation between 2D and 3D segmentations in a shared latent space. During training, 2D and 3D segmentations are registered to a common 3D template and its corresponding 2D projections, and the resulting stationary velocity fields are jointly decomposed using principal component analysis (PCA). This joint modeling allows the 2D-to-3D mapping to be learned directly from data rather than computing correspondences at inference time. For unseen subjects, the 3D shape is recovered directly by lifting the 2D latent coordinates to the 3D PCA subspace, thereby eliminating the need for iterative 3D-to-2D projection. Experiments on NMDID demonstrate that the proposed joint 2D-3D SSM outperforms a widely-used 3D-only SSM baseline while achieving inference approximately 4 times faster, at under 3 seconds per subject. The code is available at: https://github.com/florence-dellaniello-picard/joint2d3d-ssm.