Probabilistic mesh estimation improves deformable object tracking
Differentiable Mesh State Estimation via Factor Graph Inference for Deformable Object Reconstruction
RoboticsComputer Vision and Pattern Recognition
Summary
Tracking and understanding objects that change shape, like soft or bendable items, is hard for robots and simulations. The authors created a new method that uses a detailed 3D mesh made of tiny tetrahedrons to represent these objects. Their approach combines knowledge about physics, noisy sensor data, and smooth movement over time into a single calculation to figure out the object’s shape accurately. Tests on airway experiments and cube models show their method works well for objects that both move rigidly and deform. This could help robots better see and use soft or flexible things.
What this means in practice
- •For robotics engineers: Improve robot perception and manipulation by accurately estimating shapes of soft or flexible objects using a physics-informed mesh state estimation.
- •For medical simulation developers: Enhance realistic simulation of deforming anatomical structures like airways using probabilistic mesh reconstruction from sensor data.
Authors
Lidia Al-Zogbi, Fangjie Li, Samuel Tobin, James Ferguson, Nithesh Kumar, Alejandro Chara, Kuan-I Chung, Mingxing Rao, Ayberk Acar, Susheela Sharma Stern, Robert Webster, Daniel Moyer, Alan Kuntz, Caleb Rucker, Tucker Hermans, Jie Ying Wu
Abstract
Estimating deformable object states remains a fundamental challenge in robotics and simulation. We propose a novel factor graph-based framework for probabilistic mesh state estimation of deformable objects. The method directly updates a tetrahedral mesh, a rich and physically-grounded representation of an environment, by combining physics priors, noisy sensor measurements, and temporal smoothness constraints within a unified probabilistic formulation. The estimation problem is posed as a nonlinear least-squares optimization and solved using Levenberg-Marquardt. Ex vivo central-airway obstruction experiments and simulations on deforming cube models demonstrate reliable and accurate reconstruction under both rigid motion and deformation, highlighting the potential of this probabilistic approach for principled, measurement-driven mesh state estimation in deformable object reconstruction.