Automatic video modeling captures motion of flexible suture threads
Video-to-Model: Automatic Modeling of Deformable Linear Objects
Robotics
Summary
Modeling how flexible objects like suture threads move is usually complicated and requires lots of manual setup. This paper introduces a system that watches a video of a moving thread and automatically figures out how the thread moves by extracting simple parameters. The method combines tracking the thread in the video with a learned model that can then simulate the thread’s motion under new conditions. This approach can recreate realistic thread movements from just video without tedious manual tuning.
What this means in practice
- •For robotic surgeons: Generate accurate thread motion models from surgical videos to improve robotic suturing tools.
- •For animation teams: Create realistic animations of flexible threads by automatically learning motion parameters from video footage.
Authors
Akshun Sharma, Kimia Forghani, Yancy Diaz-Mercado
Abstract
This paper presents a video-to-model framework for automatically modeling the motion of a deformable suture thread from an input video. We utilize a recently developed CBF--CLF--QP numerical model that simplifies the characterization of deformable string motion through the selection of a small number of parameters. A perception module first localizes and tracks the thread in video, producing an ordered sequence of thread nodes. The observed thread motion is then processed by a spatio-temporal CNN network that estimates the effective parameters of a structured CBF--CLF--QP model. These parameters are used to simulate the thread under a user-defined needle velocity input. Experiments using unseen thread configurations and motion demonstrate that the framework can reliably reconstruct the thread behavior from video, automatically configure the structured model, and reproduce the expected thread motion with low tracking error. The proposed approach reduces the need for manual parameter tuning and provides a step toward automatic video-based modeling of deformable linear objects.