Vision language action models struggle to preserve objects during manipulation
ManiPhysicsBench: Physics-Based Assessment of Object Preservation in VLA Manipulation
Robotics
Summary
Robots that see, understand language, and move to pick things up often succeed at tasks but accidentally damage objects. The researchers created a library of 3D objects with real-world properties to test how well these robot models avoid breaking or deforming things while manipulating them. Their tests show many models finish tasks but cause damage, often gripping too simply. When retrained with detailed gripping instructions for each object, models became gentler but less successful overall and struggled to generalize their careful behavior to new objects.
What this means in practice
- •For robotics engineers: Evaluate and improve robot grasping techniques to reduce object damage during manipulation tasks.
- •For robotics simulation developers: Incorporate realistic object material properties and damage modeling into simulation platforms for safer robot training.
- •For industrial automation teams: Use physics-based benchmarks to select and train robot controllers that preserve delicate objects in manufacturing lines.$Commercial implications: Enables development and sale of safer robotic handling systems that minimize costly object damage in production.
Authors
Sangwu Park, Yeonjun In, Wonjoong Kim, Sungwon Kim, Sein Kim, Chanyoung Park
Abstract
Vision-language-action (VLA) models aim to perform diverse manipulation tasks, but task success in existing rigid-body benchmarks does not indicate whether they preserve objects. We introduce ManiPhysicsZoo, which consolidates literature-supported material properties, 3D meshes, and supporting references into reusable object assets. Using these assets, a solver-based assessment computes grasp-specific damage thresholds from object geometry, material properties, and recorded grasp conditions and compares them with recorded contact forces to assess potential deformation and fracture. Building on these components, ManiPhysicsBench evaluates object preservation in LIBERO and SimplerEnv across three physics axes and three difficulty levels. Public VLA checkpoints show a substantial gap between task success and safe success, defined as task completion while preserving the object. Their gripper commands concentrate near full opening and closure, with largely similar aggregate distributions across objects, consistent with binary gripper supervision. We examine how object-specific continuous gripper labels change model behavior by retraining a VLA model. The retrained model shows more object-dependent gripping and higher safe success, but lower task success and limited generalization of object-preserving behavior.