RecastVLA improves robot task success by tracking past actions
RecastVLA: From Past Interaction to Future Control with Adaptive Policy States
Robotics
Summary
Robots often need to remember what they did before to do complex tasks correctly, even when the current view doesn't show previous steps. The authors designed a system called RecastVLA that lets a robot keep an internal memory of past actions using a special way to adjust its decision-making process as it acts. This memory helps the robot plan better future moves without needing extra expert guidance during use. Tests on multiple robot task sets showed RecastVLA leads to higher success rates than similar robots without this memory feature.
What this means in practice
- •For robotics engineers: Develop robots that better remember and adapt their manipulation steps for improved task success in varied environments.
- •For automation system integrators: Integrate adaptive control policies to enhance robotic operation performance in complex, multi-step assembly or service tasks.
Authors
Wenbo Li, Jun Yang, Yiteng Chen, Wei Zhang, Qingyao Wu
Abstract
Sequential manipulation requires a robot to track what has already happened, even when the current scene no longer reveals it. Policies with explicit history representations make past interactions available as context for current decisions. We ask how action generation itself can form a persistent state for subsequent control. Building on action-side test-time training, RecastVLA maintains an adaptive policy state within a flow-matching vision-language-action policy. The state is represented by shared fast weights and remains fixed throughout action generation. Depth-specific interfaces read the same state, while features across depths and flow evaluations jointly define one update for the next policy call. Subsequent action losses train the initialization, interfaces, and update rule by differentiating through earlier state transitions. At deployment, updates use the policy's own action-generation features without expert action labels. Across LIBERO, RoboTwin, RoboDojo, and twelve real-robot tasks, RecastVLA improves mean success over a matched policy trained without test-time training, including 10.68 percentage points on RoboTwin Clean-to-Clean. In controlled RoboTwin comparisons, retaining state improves success, and the shared design exceeds independently trained layer-local TTT by 2.58 points.