Liver surgery guidance improved with scale-free 3d model alignment
Landmark-Free and Scale-Blind Multi-View Registration of Preoperative Liver Models to Intraoperative Surfaces
Computer Vision and Pattern RecognitionArtificial IntelligenceHuman-Computer Interaction
Summary
Aligning 3D models of a liver taken before surgery with the actual liver surface during surgery helps doctors work more precisely. The difficulty is that images taken during surgery often don't have a correct size scale, making alignment tricky. The authors created a method called SimCompose that figures out the right size and position automatically without needing special markers. Their system also checks if the alignment might be wrong and can fine-tune the match by allowing small, realistic deformations. Tests show this method works better than previous ones, including on real patient data.
What this means in practice
- •For surgical navigation teams: Enable more accurate alignment of preoperative liver scans with live surgical views without relying on landmarks or prior scale knowledge.
- •For medical imaging engineers: Provide a scale-aware registration method that can integrate multiple intraoperative views and detect registration failures automatically.
Authors
Jiaming Feng, Gerardo Loza Galindo, Shahid Farid, Sharib Ali
Abstract
Augmented-reality guidance in laparoscopic liver resection requires aligning preoperative 3D liver models from CT/MRI with intraoperative surfaces. Monocular reconstruction in most cases cannot guarantee scale, albeit rigid-motion registration requires scale as an input. Our proposed framework ``SimCompose'' estimates the full similarity transformation $\Sim$, including scale, without relying on landmarks and applies the same transformation across multiple views with known relative geometry. However, reliable detection of registration failures remains challenging in surgical settings, where ground-truth alignment is generally unavailable. For this, we propose a geometry-based selector that evaluates candidate transformations without requiring ground truth. Large disagreement among these transformations is used to flag potentially unreliable registrations. Finally, we propose to use a non-rigid refinement based on a low-dimensional deformation model from Neo-Hookean simulations in the Simulation Open Framework Architecture (SOFA), without additional network training. SimCompose is trained exclusively on synthetic partial views. Its encoder processes point clouds normalised to a unit root-mean-square radius, without access to metric scale, and filters each view before multi-view fusion. Evaluation on unseen synthetic data (300 single-view pairs per run; 100 three-view registrations) shows that multi-view integration and candidate selection increase the success rate under stringent $\Sim$ accuracy criteria from 38.4% for a single-view, single-method configuration to 77%. Existing learning-based baselines do not satisfy these criteria even when provided with the exact scale. On five in-vivo patients with 92 keyframes, the rigid registration stage reduces surface error more than three-fold relative to learning-based baselines with external scale information.