Force aware models improve robot control in contact rich tasks
ForceLDM: A Force-aware Latent Dynamics Model for Contact-Rich Manipulation
Robotics
Summary
Robots often struggle when they have to touch and manipulate objects because seeing alone doesn’t tell them enough about the forces involved. The authors designed a new model called ForceLDM that learns to predict how forces and movements will change in the near future, helping robots plan their actions better. They train the model using extra information about forces and motion, but the robot only needs current observations when working. This approach helps robots handle tricky tasks with better accuracy and adapt to new objects or changes in the environment.
What this means in practice
- •For robotics engineers: Develop robot control policies that better predict and respond to forces during complex object manipulation tasks.
- •For automation system integrators: Implement robust manipulation systems that maintain performance despite environmental changes or new objects.
Authors
Peiyuan Tang, Yao Li, Xiaodong Zhang, Wuyang Zhang, Qin Xia, Yanyong Zhang, Zijiang Yang
Abstract
Contact-rich manipulation requires fine-grained control and complex physical interactions with the environment, posing significant challenges for robot policy learning. Vision-only policies struggle with such tasks, as contact states and interaction forces are difficult to infer from visual observations alone. Existing methods mitigate this issue by incorporating force or tactile sensing. However, they often treat these signals as auxiliary observations without fully exploiting their role in modeling future interaction dynamics. To address this, we propose ForceLDM, a force-aware latent dynamics model that learns future contact and motion dynamics to guide action generation. Rather than directly predicting future observations, which can entangle task-relevant dynamics with irrelevant sensory details, ForceLDM learns a compact representation of future dynamics through knowledge distillation. Specifically, a teacher network with privileged future force/torque and optical flow signals extracts future dynamic features, which are then distilled into a student network trained only on current observations. This enables the student model to reason about upcoming contacts and physical dynamics without relying on future sensory inputs at deployment. To improve robustness and generalization, we further introduce a curriculum-based progressive noise injection strategy to mitigate over-reliance on future features. Experiments on five real-world contact-rich manipulation tasks demonstrate that ForceLDM outperforms state-of-the-art baselines, generalizes effectively to novel objects, and remains robust to environmental perturbations. Project page and code are available at https://tpy001.github.io/ForceLDM/.