Ultradiff enables shape optimization with differentiable ultrasound ray tracing
UltraDiff: Differentiable Ray Tracing in Ultrasound for Shape Optimization
GraphicsComputer Vision and Pattern Recognition
Summary
Medical ultrasound images are made from echoes of sound waves bouncing inside the body, but understanding exactly what shapes produce those echoes is tricky. The authors created UltraDiff, a tool that simulates ultrasound imaging in a way that can be reversed to guess the shape of internal tissues from the echoes. It starts with a simple shape and keeps improving it until the simulated ultrasound matches the real one, like solving a puzzle by trial and error. This new way works without needing pre-marked images and helps recover detailed shapes like vertebrae from ultrasound data.
What this means in practice
- •For medical imaging teams: Refine 3D models of anatomical structures directly from ultrasound scans without manual segmentation by using gradient-based shape optimization.
- •For robotic surgery developers: Improve real-time shape reconstruction of internal organs during robotic procedures by integrating differentiable ultrasound simulation into feedback control.
Authors
Felix Duelmer, Magdalena Wysocki, Nassir Navab, Mohammad Farid Azampour
Abstract
Physically-based differentiable rendering enables gradient-based optimization of scene parameters by matching rendered images to measurements, but has so far mainly focused on light transport. We extend this paradigm to medical ultrasound, where image formation resembles transient rendering: echoes are binned by time-of-flight rather than projected onto an image plane. We present UltraDiff, a modular framework for differentiable ultrasound ray tracing. UltraDiff formulates ultrasound image formation as a path-space integral, gated by travel time between the transducer and tissue interfaces, and derives a Monte Carlo estimator of both the forward model and its gradients with respect to scene parameters. We demonstrate this on an inverse geometry estimation: starting from a sphere, an SDF is optimized until simulated echoes match measured ones, recovering vertebral surfaces from simulated B-mode sweeps and from a real robotic acquisition of a spine phantom. Unlike state-of-the-art ultrasound shape reconstruction methods, which rely on pre-segmented images, our approach operates unsupervised on B-mode images through analysis-by-synthesis, while achieving competitive geometric accuracy. Implemented on top of Mitsuba 3, UltraDiff brings differentiable path tracing to a new sensing modality and provides a foundation for inverse problems in acoustic imaging.