PhysVLMM predicts how doors and drawers respond to robot pulls
PhysVLMM: Interaction-Conditioned Physical Belief Maps for Articulated Objects in Robot Scene Maps
Robotics
Summary
Robots often need to open doors and drawers, but current maps only show objects and if they moved, not how they react to being pulled. The authors introduce PhysVLMM, a method that keeps a physical belief about each movable part and updates it with each pull observed. This lets a robot guess whether a pull will succeed, handling uncertainty and sharing info between similar parts. Tested on real homes, this method is faster and more accurate than previous ones in predicting how parts move when pulled.
What this means in practice
- •For robotics system integrators: Enable robots to predict and plan door and drawer opening actions with uncertainty-aware physical models for better manipulation success.
- •For home automation engineers: Improve robot interaction with household articulated objects by sharing observed pulling dynamics across similar parts to speed up adaptation.
Authors
Dibyendu Ghosh
Abstract
Mobile robots must open doors and drawers, yet their scene maps record what each object is and whether it has moved, not how it will respond to a pull. This paper proposes PhysVLMM, a method that maintains a physical belief for every articulated part of a Vision-Language-Motion Map and answers a planner's queries about it, with two algorithms: a map update and a query. Watching a part move provably cannot identify its dynamics, and a measured pull identifies only the pulled part, up to one direction. PhysVLMM therefore (i) keeps that direction as a posterior instead of a point estimate, (ii) passes the evidence of each pull in closed form to the untouched look-alike parts of the map, and (iii) answers whether an action will succeed at a stated risk, or abstains. On real doors and drawers in 18 homes, it identifies the joint type in 86% of held-out pulls (48% for a learned single-view method) and predicts the response to a pull with 41% lower error than differentiable simulation (CRPS 0.323 vs. 0.544), four orders of magnitude faster (13 ms vs. 230 s per case). The posterior lowers the final-position error from 0.460 to 0.270, sharing lowers the force error of untouched parts from 56% to 42%, and at every stated risk from 0.02 to 0.30 the false-safe answers stay within it. Pooling all measured parts of a home does as well as the look-alike groups, and simpler variants predict where a pulled part stops at least as well.