Law system integrates language and vision for better robot tasks
LAW: Unifying Language, Action, and World Modeling for Agentic Physical Intelligence
Robotics
Summary
Robots need to think in language and see the world to complete complicated tasks over time. The LAW system combines language understanding, action planning, and visual memory to help robots plan subtasks and track progress better. It uses a mix of expert models to predict next actions and assess how well the robot is doing. In tests, LAW achieved high success rates on simulated and real robot tasks, outperforming previous methods.
What this means in practice
- •For robotics engineers: Build robots that plan and execute complex physical tasks by integrating language instructions with visual feedback for better subtask management.
- •For automation system designers: Design automated systems that adapt their actions based on ongoing assessment of task progress using unified language and vision models.
- •For manufacturing operators: Improve robotic assembly lines by deploying systems that understand and balance subtasks through combined language and visual cues.$Commercial implications: Enables sales of advanced robot controllers that improve efficiency and reliability in manufacturing by integrating task comprehension and execution.
Authors
Jiajun Cao, Kai Tang, Bohua Zhang, Zezhong Qian, Ying Li, Xiaozhu Ju, Xiaobao Wei, Jiaming Liu, Yang Ding, Sixiang Chen, Yiding Sun, Siyuan Qian, Tinghao Wang, Jixian Wu, Yichen Guo, Yi Zhang, Shilong Zou, Yong Dai, Jian Tang, Shanghang Zhang
Abstract
World-action models jointly predict actions and future world states, but long-horizon manipulation further requires language reasoning to plan subtask objectives, assess execution progress, and guide action selection. We present LAW, a closed-loop VLM--WAM framework that unifies language reasoning, action generation, and world modeling through semantic subtask progress. Within the LAW agent, a hybrid-expert vision-language model plans the remaining subtasks, estimates the values of and fuses actions proposed by a world-action model, and asynchronously assesses execution status to support continued execution and subtask transitions. Within the world-action model, Mixture-of-Transformer Experts integrates modality-specific and shared computation for joint video--action modeling. Dual-stream hierarchical visual memory maintains temporal context for both models, while subtask-local value learning aligns action evaluation with semantic objectives and preserves supervision from successful prefixes of failed trajectories. LAW achieves 94.21% average success across 50 RoboTwin tasks. Replacing the WAM backbone with OpenWAM-$alpha$ further raises the average success rate to 95.56%, establishing a new open-source state of the art on RoboTwin. Despite having only 2B parameters, our VLM also outperforms variants using Embodied-R1.5 and GPT-5.5 High. On three long-horizon real-robot tasks, LAW achieves 65% average success, compared with 28.33% for $pi_{0.5}$ and 25% for Fast-WAM.