DSWAM: A Dual-System World Action Foundation Model for Fine-Grained Robot Manipulation
2026-07-06 • Robotics
RoboticsArtificial Intelligence
AI summaryⓘ
The authors introduce DSWAM, a robot control system that combines two approaches: a fast action executor (System 1) and a slower language-based planner (System 2) for breaking down complex tasks into smaller steps. This helps robots handle multi-step household tasks more effectively by translating broad commands into detailed actions. They also compare their method fairly with existing vision-language-action policies using the same robot and data. To make it work smoothly on real robots, they optimize the system for speed and responsiveness during execution.
World Action ModelsVision-Language-Action PoliciesRobot ManipulationTask DecompositionVision-Language PlanningReal-Robot EvaluationTensorRTAsynchronous ExecutionDeformable Object ManipulationVideo-Based Supervision
Authors
Jian Zhu, Jianjun Zhang, Taiyi Su, Tianbin Liu, Zhangyuan Wang, Kai Xie, Zitai Huang, Chong Ma, Youzhang He, Tianjian Wang, Hanyang Wang, Weihao Ding, Yi Xu
Abstract
World Action Models (WAMs) provide a promising alternative to Vision-Language-Action (VLA) policies by using video-based world modeling as dense supervision for robot action learning. Existing WAMs excel at physically grounded execution, but typically lack the explicit language-level planning interface in VLM-based VLAs for decomposing coarse instructions. Such decomposition becomes important when household tasks involve complex multi-step goals, where coarse user commands need to be converted into sequences of fine-grained executable subtasks. Meanwhile, the field still lacks a fair real-robot comparison between VLA and WAM execution capabilities, since existing systems often differ in data, robot embodiments, and task protocols. To address both the decomposition gap and the need for a controlled WAM-VLA comparison, we introduce DSWAM, a Dual-System World Action Foundation Model for fine-grained robot manipulation. DSWAM keeps a System 1 WAM executor as the default control path and optionally activates a System 2 vision-language subtask planner only when task decomposition is useful. The planner predicts executable subtasks from short-term visual history and a global task prompt, while the WAM executor performs world-aware action generation for each instruction or subtask. The executor is trained with action prediction and video co-training, but inference directly predicts action chunks without explicit future video generation. To make this execution path practical on real robots, we further integrate TensorRT acceleration, asynchronous execution, and real-time chunking (RTC) so that policy queries do not block robot control. To provide a fair real-robot comparison with VLA policies, we build and evaluate DSWAM under the DeMaVLA real-world deformable manipulation setting with matched robot platform, pretraining data, post-training data, and evaluation criteria.