AffordTrajDP: Dynamic Affordance-Guided Visuomotor Policy Learning for Robotic Manipulation
2026-08-03 • Robotics
Robotics
AI summaryⓘ
The authors propose a method called AffordTrajDP that helps robots handle objects more accurately by creating a moving guide based on where and how to touch the object over time. Unlike older methods that use fixed contact points which can cause errors if the object moves, their approach updates the guidance dynamically by tracking the object's position and orientation. This makes the robot's actions more precise and consistent, especially in tasks requiring careful handling. They tested their method in both simulations and real robots, showing improved success rates and robustness to changes in object placement and appearance.
AffordanceImitation LearningRobotic ManipulationRGB-D ObservationSE(3) PoseTrajectory PropagationEnd-EffectorManiSkill3Real-World RoboticsTemporal Consistency
Authors
Gaoyuan Wu, Ziyu Shan, Haoyang Du, Yuyao Jiang, Ziwei Wang
Abstract
Affordance-guided imitation learning has shown impressive performance in robotic manipulation tasks by compressing visual perception into task-specific geometric constraints (e.g., fixed contact points). However, the commonly used static affordances can become inconsistent in precision-critical tasks or under object location perturbations, leading to post-contact trajectory drift. To address this issue, we propose AffordTrajDP, a dynamic framework that constructs affordance trajectories via object-centric temporal propagation to guide the progressive manipulation process. Specifically, given an RGB-D observation, our core insight is that a retrieved anchor affordance, which captures the desired contact point between the end-effector and the target object, can be propagated forward via affordance propagation, using the object's SE(3) pose as a natural propagation medium, to yield an affordance trajectory that provides temporally consistent, state-aware guidance throughout execution. AffordTrajDP achieves 70.0% average success rate on ManiSkill3, outperforming strong baselines by up to 17.8%. Real-world experiments on Galaxea A1 and UR7e robotic arms, covering StackCube, PickCup, AdapterInsertion, Ring-on-Peg, Put-in-Bowl, and USB Insertion, further validate robustness under object placement variations and appearance changes, with seen and unseen object instances evaluated on Galaxea A1, and ablations confirm the contribution of each proposed component.