Active perception improves robot manipulation across viewpoints
ActiveScale: Scaling Active Perception for Robots across Model, Data, and Hardware
RoboticsMachine Learning
Summary
Robots often miss important details when they look at things from only one angle, which makes it hard for them to do tasks well. The authors created ActiveScale, a system that helps robots actively change their viewpoint and remember previous views to better understand what they see. They trained the system using lots of video data where humans naturally move, so the robot learns to handle changing views. They also built a robot platform to collect examples of moving and manipulating things at the same time. Their tests show this approach helps robots succeed more often at tasks needing active perception.
What this means in practice
- •For robotic system developers: Enable robots to actively move cameras during tasks to improve manipulation success by coordinating vision and action across viewpoints.
- •For robotic teleoperation operators: Collect synchronized demonstrations of robot viewpoint changes and manipulation with a single operator to train better active perception models.
- •For automated warehouse teams: Improve robots' ability to handle occluded or hidden items by integrating active perception models that use multiple viewpoints effectively.$Commercial implications: Improved active perception enables warehouse robots to reliably pick and sort items, making robotic automation more viable and marketable.
Authors
Shuai Zhou, Kaisheng Pang, Wenxuan Song, Wenjie Zhang, Xinhu Zheng, Haoang Li
Abstract
Active perception is essential for robotic manipulation when fixed viewpoints leave task-relevant information occluded or unobserved. However, enabling vision-language-action (VLA) models to reason across changing viewpoints and actively acquire informative observations remains challenging. We present ActiveScale, a framework that advances active perception through coordinated model, data, and hardware designs. Our model augments a VLA with historical video observations and explicit camera-pose supervision, using per-frame pose tokens and a lightweight prediction head to associate observations across viewpoints and support a coherent understanding of the scene. To learn from the camera motion naturally present in human activity, we introduce a scalable human--robot mid-training recipe using 1000 hours of egocentric and robotic data, adapting the model to temporal inputs and pose supervision. We further introduce Active-perception Mobile-manipulation Platform (AMP), a robotic platform that supports active perception and mobile manipulation through single-operator teleoperation, enabling scalable collection of demonstrations that coordinate viewpoint changes and manipulation. Experiments demonstrate improved success rates on active-perception tasks, while ablation studies validate the contributions of camera-pose-aware modeling and egocentric mid-training. Together, these components provide an integrated foundation for studying and developing active perception in robotic manipulation.