Vis2Reg: Visibility-Aware Landmark-Free Geometric 3D--2D Registration for Liver Laparoscopy
2026-07-20 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial IntelligenceHuman-Computer Interaction
AI summaryⓘ
The authors developed Vis2Reg, a method to better match a detailed 3D liver model with partial 2D views seen during surgery, even when parts are hidden. They use a clever way to focus only on visible areas, helping the system learn without needing extra labeled data. Their approach combines a first step that roughly lines up the models with a second step that fine-tunes the fit using a special neural network. Tests show Vis2Reg is both accurate and fast for real surgery images.
3D-2D registrationliver modelingaugmented realityself-supervised learninggeometric alignmentdeformation fielddifferentiable rasterizationChamfer DistanceDice scorelaparoscopic surgery
Authors
Jiaming Feng, Xukun Zhang, Shahid Farid, Sharib Ali
Abstract
Accurate 3D--2D liver registration, which aligns preoperative 3D models to partial, view-dependent intraoperative surface observations, is critical for AR-guided laparoscopic surgery but remains challenging due to severe occlusion, limited visibility, and the lack of 3D ground-truth supervision. Existing landmark-free approaches perform partial-to-complete geometric alignment, yet robust self-supervision under extreme partial visibility remains difficult. We propose Vis2Reg, a visibility-aware registration framework that explicitly constrains deformation using mask-consistent visible regions. We introduce a visibility-aware self-supervision that derives a visible-domain 3D supervision signal from intraoperative masks, enabled by differentiable point rasterization and mask-guided back-projection. This formulation improves robustness under severe occlusion while maintaining fully self-supervised learning. Vis2Reg combines a robust geometric rigid initialization module with an implicit neural deformation field for stable alignment. Vis2Reg achieves a Dice score of 92.6\% and a Chamfer Distance of 1.43 mm on real intraoperative datasets, with 111 ms per-frame inference time, demonstrating both accuracy and practical efficiency.