Focus pooling improves robot vision for control from fewer data
Localized Visual Feature Aggregation via Focus Pooling for Visuomotor Policies
Robotics
Summary
Robots that learn from cameras often struggle because they get distracted by unnecessary details in images. The authors found a way to help robots pay attention only to the important parts of what they see that affect their actions. They created a method called FocusPool that picks out relevant visual information based on what the robot senses about its own movements, making learning more efficient and accurate. This approach works better than previous ways to select important features and helps robots perform tasks more successfully.
What this means in practice
- •For robotics engineers: Improve robot task skills by using FocusPool to efficiently learn from visual and proprioceptive cues.
- •For industrial automation teams: Deploy advanced visual attention techniques to increase robot success rates in manipulating objects in manufacturing.
Authors
Ruiyu Wang, Zheyu Zhuang, Danica Kragic, Florian T. Pokorny
Abstract
Focusing on spatially localized, control-relevant visual cues has been shown to improve data efficiency in visuomotor policies by reducing the need to model task-irrelevant visual variation. Existing methods often impose this focus through input preprocessing, such as cropping control- or object-centric regions in RGB images or point-clouds. However, it remains underexplored whether such localized features can be exposed directly from commonly used convolutional neural network (CNN) encoded features. In this paper, we show that intermediate CNN features preserve localized visual context for control, but existing pooling methods fail to aggregate it effectively. We introduce FocusPool, an attention pooling module that selectively aggregates intermediate visual features according to their relevance to the robot's current proprioceptive context. The resulting pooled representation captures task-progressive, control-relevant local information and is used directly for policy learning. Across simulation and real-world experiments, FocusPool improves policy success rates over pooling and explicit local focus methods by 36.2% and 41.2%, with training only 5.8% of encoder parameters.