KAM-WM: Kinematic Affordance Maps from Latent World Models for Robot Manipulation
2026-07-06 • Robotics
Robotics
AI summaryⓘ
The authors created a method called KAM-WM that teaches robots how to interact with objects using very few examples. Instead of just knowing where to touch, their system also gets a rough idea of how the robot should begin moving. They do this by using a pre-trained video model to get cues about direction and motion without needing extra training. Their method helps robots do tasks more successfully compared to simpler approaches that only know where to touch. This shows that frozen video models can give helpful hints for robot control without extra computation at test time.
Kinematic Affordance MapFlow Matchinglatent video modeldiffusion policyvisual priorsproprioceptionrobot manipulationsegmentation maskslatent velocityPerceiver
Authors
Xinyu Shao, Keru Zhou, Guowei Huang, Yajun Gao, Tongtong Cao, Xiu Li
Abstract
Learning manipulation from few demonstrations requires visual priors that capture not only where to interact, but also how the interaction should begin; static priors such as segmentation masks encode only the former. We present KAM-WM, a framework that extracts a coarse directional interaction cue from a frozen latent video world model without rollout or world-model fine-tuning. KAM-WM queries a Flow Matching image-to-video backbone once and interprets its single-step latent velocity as a Kinematic Affordance Map (KAM), which provides task-conditioned interaction regions and coarse motion structure. A lightweight Perceiver compresses KAM into tokens that condition a diffusion policy together with RGB observations and proprioception. Across LIBERO and RoboTwin2.0, KAM-WM reaches 90.6% average success on LIBERO and achieves 65.7% and 22.4% success rates in the Easy and Hard settings on RoboTwin2.0, respectively. Controlled comparisons against a zero-order mask prior suggest that part of the gains comes from directional information beyond spatial localization alone. These results indicate that, in the evaluated settings, a frozen video model can provide a useful first-order visual prior for control without the test-time cost of future rollout.