World action models improve task success with test-time action selection
Seeing Through the Illusion: Test-Time Scaling for World Action Models
Robotics
Summary
When robots or AI systems imagine what to do, the planned actions and imagined results don’t always match well. The authors propose a way to pick better actions at the moment decisions are made by scoring each possible action on how valid and consistent it is with the imagined motion, and how well it progresses the task. Using this method, called WISE, they improve the success rate of robots completing tasks by selecting better actions without changing the original model. They show this works well in different robot task environments.
What this means in practice
- •For robotic system developers: Improve robot action selection accuracy in dynamic tasks by scoring and choosing better candidate actions at execution time without retraining models.
- •For automation engineers: Enhance performance of automated systems by integrating evaluation-based action selectors that ensure task progress and motion consistency before execution.
Authors
Cheikh Ahmed, Mahdi Mostajabdaveh, Milad Yazdani, Zirui Zhou
Abstract
World action models (WAMs) jointly generate actions and imagined futures, but a plausible prediction need not be realized by its co-generated action. We introduce WISE (\emph{World Action Inference Scaling through Evaluation}), a pre-execution selector that scores each sampled rollout on action validity and temporal regularity, inverse-dynamics consistency between the proposed action and the imagined motion, and instruction-conditioned task progress, and executes the highest-scoring candidate while keeping the WAM frozen. With four candidates per decision, WISE raises closed-loop success from 6.0\% to 24.0\% on RoboLab with Cosmos~3 Edge and from 75.1\% to 78.3\% mean success on LIBERO with DreamZero, and exceeds the best existing selector in aggregate by 4.0 and 0.9 points. Component ablations and agreement analyses show that the three scores rank candidates differently and that task progress carries the largest weight in the fused decision.