PLaCA improves robot action success by predicting future local changes
PLaCA: Learning Patch-Level Future Latent Change for Action Generation
RoboticsComputer Vision and Pattern Recognition
Summary
Robots need to understand how the things around them will change to perform tasks better. The authors created PLaCA, a new method that helps robots predict small, local changes in what they see before acting. Instead of guessing the whole future picture, PLaCA focuses on important patches that will change, which helps the robot plan its moves more accurately. This approach improved robot task success in tests while keeping the training efficient.
What this means in practice
- •For robotics engineers: Generate more precise manipulation actions by predicting local environmental changes from visual and language inputs.
- •For automation system designers: Improve sequential task execution reliability by integrating patch-level future state prediction into decision-making.
Authors
Zhan Yi, Yuchen Wang, Chao Li, Lei Hong, Boyuan Yang
Abstract
Vision-Language-Action (VLA) models generate robot actions from visual observations and language instructions, but effective manipulation also requires anticipating how the environment will change. Existing world-action models explore future image prediction and latent dynamics modeling. However, generating complete future images can be computationally expensive, while highly compressed latent representations may obscure local changes. To address these challenges, we propose PLaCA, a world-to-action framework built around future latent change. PLaCA directly predicts spatially structured patch-level future latent change from the current multimodal input, preserving local information about expected environmental changes. We introduce patch-level change-aware supervision to emphasize regions undergoing larger changes and a future-to-action interface that explicitly incorporates predicted future latent change into action generation. Experiments on LIBERO-Long show that PLaCA achieves a 79.0% success rate, improving upon the Action-only baseline by 5.0 percentage points. On CALVIN, PLaCA improves the average sequence length from 2.784 to 3.564. These improvements are achieved while updating only 2.95% of the model parameters during the second training stage, demonstrating the effectiveness of future latent change modeling for downstream robotic manipulation.