Self improving models learn actions from videos without human input
Self-Improving Video-to-Action Models
Artificial IntelligenceComputer Vision and Pattern Recognition
Summary
Making robots or AI agents that learn how to act by watching videos usually needs a lot of human-labeled examples showing what actions to take. This paper introduces Mirror, a method that teaches an AI to figure out actions by trying to copy what it sees in a video and then learns from its own attempts. The AI improves itself over time without anyone telling it the right actions or giving rewards. This approach worked better than others on game tasks like Atari and Minecraft, even reaching expert-level results.
What this means in practice
- •For robotics engineers: Create robots that learn to perform tasks by watching videos rather than needing large labeled datasets of actions.
- •For video game ai developers: Build game agents that improve at playing complex tasks through self-guided learning from video observations in games like Minecraft.
Authors
Hyunwoo Ryu, Eric Ming Chen, Max Wilcoxson, Vincent Sitzmann
Abstract
Video generative models are an emerging tool for embodied agents: they can serve as scalable visual planners, generating videos that depict how to complete previously unseen tasks. However, executing these plans typically requires a video-to-action or "inverse dynamics" model trained on large, human-collected video-action datasets, severely limiting scalability. We introduce Mirror, a self-supervised approach that learns video-to-action mappings through environment interactions, without prior action data or hand-crafted rewards. Mirror learns by attempting to follow a visual plan by executing its best guess of the underlying actions in the environment. Mirror then trains on the resulting video-action pairs, incrementally self-improving and following the visual planner more closely. Across Atari, OGBench, and DeepMind Control, Mirror outperforms leading learning-from-observation methods with learned video plans, and achieves near-expert performance with oracle plans. On Minecraft BASALT, it outperforms methods with access to expert actions or hand-engineered task knowledge. In Minecraft survival, it is the first method to acquire an iron pickaxe through self-improvement alone, without rewards or expert action labels. In the offline setting, with expert action labels but no environment interaction, it sets a new state of the art in obtaining diamonds. Our results suggest that advances in video models can translate into increasingly capable agents without a corresponding need for hand-crafted rewards or large collections of expert action data.