Vision language action models improve robot tasks during camera outages
Learning to Act under Visual Interruptions with Vision-Language-Action Models
RoboticsArtificial Intelligence
Summary
Robots that see and act based on camera input can struggle if a camera stops working mid-task. The authors study how losing camera views affects robot actions and create a test called MAIL-Bench to measure this. They also develop MINT, a way to help robots keep working by guessing missing visual information. Their experiments show that MINT helps robots succeed more often when cameras fail, even on real robots.
What this means in practice
- •For robotics engineers: Maintain robot manipulation performance when camera views fail by using models that predict or extrapolate missing visual input.
- •For industrial automation teams: Deploy robots in environments with unreliable cameras by integrating systems that handle visual interruptions to reduce task failures.
Authors
Mingle Jiang, Rui Xu, Yunke Wang, Chang Xu
Abstract
Vision-language-action (VLA) models have demonstrated strong capabilities in robotic manipulation, but they are typically developed and evaluated with all camera streams available throughout task execution. When a camera stops delivering frames during task execution, the policy must continue acting without access to subsequent observations from the missing view. Despite its practical importance, how such interruptions affect closed-loop manipulation remains insufficiently understood. To investigate this problem, we introduce MAIL-Bench, a benchmark that evaluates visual interruptions with VLA models. By interrupting different cameras at multiple stages of each policy's successful reference trajectory, MAIL-Bench measures how well policies retain their capabilities when visual inputs become unavailable. Building on this benchmark, we propose MINT, which first trains VLA policies to remain functional under missing visual inputs. At inference time, MINT selectively supplements missing observations using optical-flow extrapolation or an action-conditioned world model, and withdraws predicted views when they become unreliable. Experiments on $π_{0.5}$ and GR00T N1.5 show that MINT significantly improves task success under camera loss over the original models. Experiments on AgiBot G2 further demonstrate the real-robot deployment under camera loss. The benchmark is available at https://minglejiang.github.io/Mail-Bench/