Observation anchored forecasting improves tissue deformation prediction
Observation-Referenced Tissue Deformation Forecasting with Tool-Driven and Residual Dynamics
Robotics
Summary
Predicting how soft tissues change shape when surgical tools move is important for planning surgeries and advanced robotic assistance. This paper introduces a new way to predict tissue movement by referencing each point to its last seen position and adding predicted changes from known tool movements. This approach reduces errors from mismatches in initial setups and ongoing simulation issues. The authors also add a model to capture repetitive tissue motions that tools don't cause. Their tests on surgical videos show this method improves accuracy, making it useful for future surgical planning and simulation tools.
What this means in practice
- •For surgical robotics teams: Improve robotic surgery systems by better predicting tissue responses to planned instrument motions using observation-referenced forecasting.
- •For medical simulation developers: Create more realistic digital twins and training simulators by incorporating tissue deformation predictions that adjust based on observed tool-tissue interactions.
Authors
Tom Olesch, Nan Xiao, Xiongye Xiao, Jonathan C. DeLong, Fei Liu
Abstract
Forecasting tissue deformation from future instrument motion could support predictive planning and increasingly autonomous surgical systems. We study this problem using a scene-specific differentiable surface model fitted to an observed tool-tissue interaction window. Rather than forecasting absolute simulated positions, we introduce an \textit{observation-referenced} formulation that anchors each tissue point to its last valid observation and adds only the deformation induced by the known, preplanned future tool trajectory. This reduces sensitivity to reconstruction, initialization, and persistent simulation offsets while preserving the predicted deformation response. The physical model uses differentiable, mesh-free position-based dynamics with scene-specific rigidity, attachment, region scale, and grasp coupling. We further introduce an optional \textit{observation-referenced residual model} combining graph-spectral spatial modes with periodic harmonic dynamics for structured motion not explained by the tool input. We evaluate the framework retrospectively on 12 surgical manipulation clips from three datasets. Observation referencing provides the largest gain, with a median within-clip error ratio of $0.598$ relative to the corresponding absolute simulation forecast. The optional periodic residual further improves forecasting when the remaining motion exhibits periodic structure. These results support observation-referenced deformation prediction as a practical building block for future surgical world models, digital twins, and generative simulators that must predict how tissue evolves under instrument actions.