Object focused visuomotor control improves robotic manipulation robustness
Object-Centric Conditioning for Visuomotor Flow Matching
Robotics
Summary
Robots that use vision to guide their movements can get confused by changes in what they see or distractions in the background. The authors created a new method called SlotFlow that helps robots focus on important objects instead of irrelevant details. This approach separates what the object is from where it is, making the robot’s control more reliable even when things change or look different. Their tests show this method works better than older ones, especially when things in the scene move or look unusual.
What this means in practice
- •For robotics engineers: Develop robotic arms that maintain precise control under changing object locations and visual noise.
- •For industrial automation teams: Improve factory robot manipulators’ reliability when facing cluttered or shifted work environments.
Authors
Jijie Li, Xu Yang, Junhong Zou, Chunhai Zhao, Chaoyang Zhao, Zhen Lei, Xiangyu Zhu
Abstract
Robot visuomotor policies are commonly formulated as autoregressive, diffusion-based, or more recently, flow matching models. Among them, Action-to-Action (A2A) flow matching improves inference efficiency by initializing generation from historical action priors rather than stochastic noise. However, stale historical motion patterns and entangled global visual representations can jointly reduce robustness under spatial out-of-distribution (OOD) shifts and visual distractors. In this work, we propose SlotFlow, an object-centric flow matching policy for robust visuomotor manipulation. SlotFlow decouples scene observations into semantic ("what") features and lightweight image-plane spatial ("where") cues to provide object-aware policy conditioning and current-state grounding. The semantic representation suppresses irrelevant background correlations, while the spatial cue improves adaptation to shifted object configurations. Extensive simulation and real-world experiments demonstrate improved robustness under visual distractors and severe spatial perturbations while preserving the low-step inference efficiency of A2A. Controlled initialization and perception ablations further identify object-centric grounding as a major source of the gains and show that it complements, rather than replaces, useful historical motion priors.