Geometry-Aware Motion Latents for Learning Robust Manipulation Policies
2026-07-06 • Robotics
RoboticsArtificial Intelligence
AI summaryⓘ
The authors present GeoMoLa, a method for teaching robots how objects move by focusing on changes in 3D shapes over time instead of just looking at camera images. By predicting how point clouds (3D data points) shift rather than reconstructing visuals, their approach better captures real physical movement. This method works well even with just one camera view and performs better than previous techniques that needed multiple views. They show that understanding spatial changes is key for robot manipulation and that their learned motion codes can apply to new situations reliably. Real-world tests confirm that this geometric focus helps robots manipulate objects accurately with minimal practice.
motion latentsrobotic manipulationpoint clouds3D geometryRGB-D inputspatial understandingmotion abstractiongeometric transformationssingle-view inputphysical motion prediction
Authors
Yunchao Zhang, Yijia Weng, Ruizhe Liu, Ming Hu, Leonidas Guibas, Yanchao Yang
Abstract
Learning motion latents for robotic manipulation heavily relies on extracting motion patterns from visual sequences, yet effective action abstractions require understanding three-dimensional geometric transformations. Here, we introduce GeoMoLa (Geometry-Aware Motion Latents), which learns discrete motion latent codes by predicting how point clouds evolve during manipulation rather than reconstructing visual observations. This four-dimensional objective -- spatial geometry changing through time -- forces latent representations to encode actual physical motion rather than appearance patterns. GeoMoLa achieves state-of-the-art performance using only single-view RGB-D input, while existing methods require multi-view reconstruction, succeeding across diverse manipulation benchmarks. Our ablations reveal that geometric prediction is the key to driving performance, quantitatively validating that manipulation depends on spatial understanding. Furthermore, the learned codes exhibit effective motion abstraction: applying them to novel scenes produces physically consistent transformations regardless of visual context. Our real-world experiments also confirm this robustness capability, achieving robust manipulation with minimal demonstrations in cluttered environments where geometric reasoning determines success. Thus, we demonstrate that effective motion latents for robot control can better emerge from understanding motion through its three-dimensional effects rather than pixel-level patterns.