LaST05 improves robot manipulation by focusing on interactions
LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning
Robotics
Summary
Robots often struggle to understand how they should move when interacting with objects because they look at the whole scene instead of the close-up details where they touch things. The authors created LaST0.5, a new method that helps robots pay special attention to how they interact with objects, making their movements more accurate. This method combines different parts to better predict what will happen and when to make adjustments during tasks. Tested with a huge amount of robot data and real-world tasks, LaST0.5 performs very well, especially in long and delicate robot actions.
What this means in practice
- •For robotics engineers: Develop robot controllers that better predict and adapt actions based on detailed robot-object interaction modeling for more precise manipulation.
- •For automation system integrators: Integrate improved action models like LaST0.5 to enhance assembly line robots’ ability to handle complex and delicate tasks reliably.
Authors
Xuheng Zhang, Yiming Zhang, Jialin Gao, Hao Chen, Jiaming Liu, Ji Li, Runzhong Huang, Yinxi Wang, Chenyang Gu, Zihan Yang, Yaowei Liu, Zhonghao Yan, Ziyu Guo, Renrui Zhang, Peng Jia, Pheng-Ann Heng, Shanghang Zhang
Abstract
World Action Models (WAMs) have recently emerged as a promising paradigm for robotic manipulation, endowing policies with strong capabilities for modeling temporal and physical dynamics. Existing WAMs, encompassing both video-first action prediction and joint video-action modeling, directly transfer global physical representations from world models to action generation. However, global scene representations may underemphasize local robot-object interactions, creating a mismatch between representations optimized for world evolution and those required for manipulation. To bridge this gap, we propose LaST0.5, a novel WAM that introduces an interaction-centric physical representation between world modeling and action generation. Specifically, LaST0.5 learns a compact interaction-centric latent space under the supervision of visual interaction cues between the robot and the environment. This reasoning space is instantiated as a latent expert that bridges the world expert and the action expert within a Mixture-of-Transformers architecture. Furthermore, LaST0.5 performs adaptive world-to-action transfer by jointly predicting future representations and progress values, using progress consistency to adjust action execution and trigger future resampling. Finally, we pretrain LaST0.5 on a large-scale robotic data corpus comprising over 14k hours of data, establishing interaction-centric physical reasoning as a scalable and effective paradigm. LaST0.5 achieves 99.5% on LIBERO, surpasses the previous state of the art on ten evaluated RLBench tasks, and demonstrates robust long-horizon execution and precise dexterous manipulation in real-world experiments.