Joint 2D and 3D shape model speeds femur reconstruction from X-rays
A Joint 2D-3D Statistical Shape Model for Orthopedic Reconstruction
Computer Vision and Pattern Recognition
Summary
Reconstructing a 3D model of the thigh bone (femur) from X-ray images is hard because X-rays only show flat pictures without depth. The authors designed a new model that learns how 2D X-ray shapes and 3D bone shapes vary together, so it can quickly guess the 3D shape from the 2D image. This approach is faster and more accurate than previous methods that try to match 3D shapes to 2D images step-by-step. Their method was tested on a real medical image dataset and performed better and quicker than common techniques.
What this means in practice
- •For orthopedic surgeons: Generate accurate 3D models of femurs from standard X-rays to improve surgical planning and implant selection.
- •For medical device designers: Use faster 3D bone reconstructions from 2D images to design implants that better match patient anatomy.
Authors
Florence Dell'Aniello Picard, Pranav Poudel, Nairouz Shehata, Frédéric Lavoie, Herve Lombaert
Abstract
Three-dimensional femoral reconstruction from radiographs supports surgical planning, implant sizing, and post-operative follow-up, but remains ill-posed as X-ray projections discard depth information. Existing methods often incorporate a 3D statistical shape model (SSM) as a shape prior to guide reconstructions toward anatomically plausible shapes, relying on iterative 3D-to-2D projection matching. Yet, these approaches are computationally expensive and constrain their SSM to a single dimensionality, leaving the statistical relationship between 2D observations and 3D geometry largely unexploited and unexplored. We instead propose a joint 2D-3D SSM that explicitly captures the co-variation between 2D and 3D segmentations in a shared latent space. During training, 2D and 3D segmentations are registered to a common 3D template and its corresponding 2D projections, and the resulting stationary velocity fields are jointly decomposed using principal component analysis (PCA). This joint modeling allows the 2D-to-3D mapping to be learned directly from data rather than computing correspondences at inference time. For unseen subjects, the 3D shape is recovered directly by lifting the 2D latent coordinates to the 3D PCA subspace, thereby eliminating the need for iterative 3D-to-2D projection. Experiments on NMDID demonstrate that the proposed joint 2D-3D SSM outperforms a widely-used 3D-only SSM baseline while achieving inference approximately 4 times faster, at under 3 seconds per subject. The code is available at: https://github.com/florence-dellaniello-picard/joint2d3d-ssm.