Robot model improves task flexibility by sharing intent across systems
PINT: A Unified Model with Shared Predictive Intent for Robot Manipulation
Robotics
Summary
Robots need to understand what they should do and how the world might change as they act. The authors developed PINT, a model that combines language-based understanding, video prediction, and action-planning tools in a new way. Instead of blending everything directly, PINT uses a shared ‘intent’ representation that helps each part specialize while working together. This improves how well robots can perform known tasks and handle new, different situations without extra training.
What this means in practice
- •For robotics engineers: Improve robots’ ability to generalize task execution in complex manipulation scenarios using a unified intent-based model.
- •For automation system builders: Enable more flexible control systems that coordinate semantic reasoning and prediction for robotic arms handling diverse tasks.
Authors
Xinyu Guo, Chao Jing, Li Yi, Xingyu Chen, Zhengxing Wu
Abstract
Generalist robot manipulation can benefit from pretrained vision-language models (VLMs) for semantic reasoning and video generation models (VGMs) for world modeling. However, existing unified models that directly fuse them through shared attention at multiple layers can make it harder to draw on their specialized capabilities. We introduce PINT, a unified model that coordinates semantic reasoning, world modeling, and action generation through a shared chunk-level Predictive Intent while preserving expert-specific pathways. PINT inserts learnable Intent Tokens into the VLM sequence to form this representation, conditioning both the VGM and an Action Expert (AE). It replaces only the VGM's prompt conditioning and retains the VGM at inference, where the AE reads its representations through one-way attention. Rather than prescribing what Predictive Intent should encode, PINT learns it jointly from the native prediction objectives of the VLM, VGM, and AE, which provide complementary views of the same upcoming task execution. Training uses two stages: Stage 1 freezes the VGM so that future video prediction shapes Predictive Intent with task-relevant future dynamics, while Stage 2 fixes the learned Intent and adapts the VGM and AE. Without embodied pretraining, PINT achieves strong in-distribution (ID) performance and substantially improves out-of-distribution (OOD) generalization, reaching 15.5% on RoboTwin 2.0-Hard compared with 1.8% for Fast-WAM and 3.2% for StarVLA, and 79.5% on LIBERO-Plus compared with 51.5% and 75.0%, respectively. Across five real-world bimanual tasks, PINT reaches 68% ID and 24% OOD success, compared with 35%/9% for pretrained Motus and 67%/35% for pretrained $π_{0.5}$.