Look Where It Matters: Adaptive Visual Refinement for Vision-Language-Action Models
2026-08-03 • Robotics
Robotics
AI summaryⓘ
The authors studied problems with how visual parts of VLA (vision-language-action) models focus on important image areas, which affects precise robot control. They found that some attention-related issues cause important spatial details to mix into less useful parts of the image representation. To fix this, the authors created AtVLA, which adds special tokens to better capture spatial info, improving task understanding without extra training data. They also introduced a system that zooms in on uncertain parts of the scene to refine decisions, leading to clearer robot actions and higher success rates in tests. Overall, their method helps robots better locate and interact with objects using vision.
Vision-Language-Action modelsVision TransformersAttention artifactsSpatial perceptionEmbodied policiesRegister tokensAttention rolloutUncertainty estimationAction refinementRobotic manipulation
Authors
Jin Cui, Yanbin Hu, Xinyue Long, Linkai Li, Boran Zhao, Pengju Ren
Abstract
Visual representations of VLA models remain unreliable for spatially precise robotic manipulation. We uncover that vision encoders in VLAs also exhibit attention artifacts previously documented in generic Vision Transformers, and further show that, in embodied policies, these artifacts are closely associated with spatial perception capabilities acquired during post-training. As the encoder learns task-relevant information such as object location, depth ordering, and local geometry, limited global-token capacity causes part of this information to spill into low-information patch tokens. We introduce AtVLA, a framework that inserts learnable register tokens into the visual encoder. Trained end-to-end using only embodied data and the original action objective, these registers emerge as dedicated carriers of embodied spatial information, while the remaining patch tokens recover clean and spatially faithful attention distributions crucial for precise target localization and fine-grained contact. Clean attention restores reliable localization, but cannot recover geometric details lost in low-resolution observations. AtVLA therefore couples attention rectification with uncertainty-gated local refinement. The action expert samples multiple action chunks and estimates uncertainty from their disagreement; only for uncertain predictions, action-conditioned attention rollout identifies the task-relevant region, which is cropped, re-encoded at high resolution, and appended to the cached prefix for refined action generation. Across LIBERO, SimplerEnv, and a challenging single-view real-world benchmark, AtVLA improves the average LIBERO success rate from 94.2% to 98.4% and real-world success from 46.5% to 69.0%. The cropping is triggered on approximately 30% of replanning steps, resulting in only 1.4-1.6x the total computation of the base model under the representative deployment setting.