VerifLM generates verified object poses from edited images for manipulation
VerifLM: Physically Verified Image-Edit-Guided Goal Pose Generation for Robotic Manipulation
Robotics
Summary
It can be hard to tell a robot exactly how to move an object using numbers alone. The authors present VerifLM, a system that starts from a picture that someone has edited to show a desired goal, then figures out the exact 3D way the objects should be placed. It uses AI language models to help guess pose constraints and a physics simulator to test if these guesses make sense in the real world. VerifLM checks multiple versions of the objects to make sure the generated positions will work for the task, helping robots stack, arrange, or place objects precisely.
What this means in practice
- •For robotic system integrators: Convert visual task goals into verified 3D object poses without requiring exact object models for robotic planning.
- •For manufacturing automation teams: Improve robotic assembly and placement tasks that need precise object orientation and contact by confirming goal poses with physical simulation.
Authors
Xuanyou Chen, Jinyu Chen, Glen Chou
Abstract
Many manipulation goals are easier to visualize than to specify numerically through explicit geometric constraints. We present VerifLM, a framework that converts image-edited goal configurations into physically verified 6-DoF object poses for planning. Using an edited image, VerifLM recovers an initial relative pose between an active object and a passive object, then prompts a vision-language model to generate axis-wise sampling constraints and a task-specific validation program. Candidate poses are evaluated in a physics simulator, and the VLM selects the most task-consistent retained candidate. To operate without predefined object meshes or physical parameters, VerifLM reconstructs metric object geometry from RGB-D observations and constructs multiple hypothetical simulations with varying physics and geometry. Parallel-world validation evaluates candidates across geometry--physics variations and ranks them by the number of worlds in which they satisfy the validation program. We evaluate VerifLM on nine manipulation tasks. VerifLM achieves the highest task success on eight of nine tasks, with the largest performance gain on goals requiring precise orientation and contact relationships.