Object interactions improve robot task learning with less data
FOCI Policy: Focus on Object-Centric Interactions for Relational Manipulation Policies
Robotics
Summary
Teaching robots how to move and interact with objects is tricky because they must learn both what to do and how objects move. The authors noticed many tasks involve short, focused interactions between objects rather than continuous action. They created a method that breaks down these tasks into smaller interaction chunks and models motion between objects instead of direct robot actions. This helps robots learn tasks more efficiently and work well in varied scenes with less training data.
What this means in practice
- •For robotics engineers: Design robots that learn manipulation tasks from fewer demonstrations by focusing on object-object interaction patterns.
- •For industrial automation teams: Improve robot adaptability in manufacturing settings by using interaction-centric policies for handling related objects with varied configurations.
Authors
Ze Fu, Pinhao Song, Yutong Hu, Renaud Detry
Abstract
Object-centric manipulation policies improve generalization by modeling object motion instead of directly predicting robot actions. However, existing methods are often limited by representations which are either too simplistic to capture interaction dynamics or too dense to learn efficiently. We observe that many rigid relational manipulation tasks are governed by short interaction phases where the relative motion between task-relevant objects is tightly constrained. Based on this observation, we propose \textsc{Foci Policy}, an interaction-centric framework that achieves a two-fold abstraction: (1) temporally, by automatically extracting compact interaction segments from demonstrations;(2) spatially, by representing skills as relative $SE(3)$ motion between task-relevant objects, yielding invariance to scene configurations and robot embodiment. Experiments on RLBench, COLOSSEUM, and real-world tasks show that \textsc{Foci Policy} achieves strong performance with substantially less training data than prior object-centric and action-centric policies. These results suggest that modeling object-object interactions provides a simple and efficient inductive bias for rigid relational manipulation. Project page: \href{https://fitz0401.github.io/foci-page/}{fitz0401.github.io/foci-page/}.