From Fixed to Free Cameras: Calibration-Free View-Robust Vision-Language-Action Model
2026-07-06 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial IntelligenceMachine LearningRobotics
AI summaryⓘ
The authors created a new robot control system called CamVLA that helps robots work well even when the camera moves or is set up differently from training. Instead of telling the robot exactly where the camera is, their system teaches the robot to figure that out on its own by separating the way it moves its hand from how it understands the camera's viewpoint. This method uses just one regular camera image and no extra depth or calibration information, making it easier to use in real-world situations. Tests in both simulations and real robot experiments showed CamVLA works better than older methods when looking from new or unknown angles.
Vision-Language-Action (VLA)camera extrinsics6-DoF hand-eye calibrationmonocular RGB imagerobot base framecamera-centric actionpose-independent controlrobot manipulationgeometric transformationviewpoint robustness
Authors
Wenhao Li, Xueying Jiang, Quanhao Qian, Deli Zhao, Shijian Lu, Gongjie Zhang, Ran Xu
Abstract
Real-world robot deployment rarely maintains the training-stage camera setup, where cameras often experience repositioning or remounting depending on actual scenarios. Existing view-robust Vision-Language-Action (VLA) policies tolerate such camera variations only when the camera extrinsics are explicitly provided, making them fragile and hard to use especially when view robustness is critical. We argue that the policy should not be told where the camera is, but rather figure it out by itself. To this end, we introduce Camera-Centric VLA (CamVLA), a new VLA model that decouples manipulation controls from camera geometry by predicting (i) a camera-centric end-effector action expressed in the local camera frame, and (ii) a 6-DoF hand-eye matrix relating cameras to the robot base. A deterministic geometric transformation composes the two predictions into a robot base-frame action. This disentangles how I should move in pose-independent camera-centric action generation from where I am looking from in camera-perspective geometric grounding. The resulting policy is calibration-free, depth-free, and single-view, requiring only a single monocular RGB image as the visual observation and task instruction at deployment. Evaluations in both simulation and real-world robot data show that CamVLA consistently improves success rates across diverse unseen viewpoints. Project page: https://alibaba-damo-academy.github.io/CamVLA/.