Endoscopic 3d reconstruction improved with dynamic texture guidance
EndoPrior-GS: Dynamic Endoscopic Reconstruction with a Joint Texture Prior
Computer Vision and Pattern Recognition
Summary
3D views inside the body during surgery help doctors operate better, but capturing these moving and changing scenes accurately is hard. The authors created a new method called EndoPrior-GS that uses smart clues about tissue appearance and shape from video frames to improve the 3D models. This method helps reduce errors in tracking movement and keeps the images clear and fast to render. It was tested on known datasets and showed substantial improvements over previous approaches.
What this means in practice
- •For robotic surgeons: Use improved 3D reconstructions during surgery to better track tissue and tools in real time with fewer visual errors.
- •For medical imaging software developers: Incorporate dynamic texture-aware priors in intraoperative imaging tools to enhance 3D model accuracy and rendering speed.
Authors
Jiaqi Huang, Shidong Wang, Tong Xin, Kabita Adhikari
Abstract
Dynamic endoscopic reconstruction is fundamental to robotic surgery and computer-assisted interventions. While 3D Gaussian Splatting (3DGS) realises real-time rendering, its application to deformable intraoperative environments remains constrained by spurious geometry and varying illuminations. To address these limitations, we introduce EndoPrior-GS, a novel pipeline that explicitly couples frame-extracted vision heuristics and estimated depth maps. EndoPrior-GS derives a joint texture prior from a tool-filtered valid tissue mask, a non-specular photometric filter, and anatomical structural salience, yielding a probability map that guides primitive initialisation and subsequent density control. The prior is further extended to the temporal domain through a texture-aware term that dynamically weighs pairwise primitive contributions during training. We conduct extensive experiments on benchmark datasets EndoNeRF and SCARED, and the obtained results show that our method EndoPrior-GS reduces Flow Error by 27.7% and 25.8% over the representative approaches while preserving competitive rendering quality and real-time rendering speed. Our project website is available at https://jiaqi-huang-77.github.io/EndoPrior-GS/.