FocusPool improves robot vision by picking out key local details
Localized Visual Feature Aggregation via Focus Pooling for Visuomotor Policies
Robotics
Summary
Robots controlling themselves with cameras often get confused by too much irrelevant visual information. The authors found that focusing on small but important parts of an image helps improve the robot’s ability to perform tasks. They developed a new method called FocusPool that smartly selects and combines visual features related to the robot’s current situation. This approach made robots better at completing tasks in tests, compared to older methods that either processed the whole image or cropped it narrowly. FocusPool also learns faster because it adjusts fewer parts of the vision system.
visuomotor policiesconvolutional neural networksvisual feature poolingattention mechanismsrobot proprioceptioncontrol-relevant visual cuesfeature aggregationrobot learningdata efficiency
Authors
Ruiyu Wang, Zheyu Zhuang, Danica Kragic, Florian T. Pokorny
Abstract
Focusing on spatially localized, control-relevant visual cues has been shown to improve data efficiency in visuomotor policies by reducing the need to model task-irrelevant visual variation. Existing methods often impose this focus through input preprocessing, such as cropping control- or object-centric regions in RGB images or point-clouds. However, it remains underexplored whether such localized features can be exposed directly from commonly used convolutional neural network (CNN) encoded features. In this paper, we show that intermediate CNN features preserve localized visual context for control, but existing pooling methods fail to aggregate it effectively. We introduce FocusPool, an attention pooling module that selectively aggregates intermediate visual features according to their relevance to the robot's current proprioceptive context. The resulting pooled representation captures task-progressive, control-relevant local information and is used directly for policy learning. Across simulation and real-world experiments, FocusPool improves policy success rates over pooling and explicit local focus methods by 36.2% and 41.2%, with training only 5.8% of encoder parameters.