Force guided robot learning improves contact task recovery actions
ForceRFT: Refining VLA Actions through Force-Guided Residual Reinforcement Learning
Robotics
Summary
Robots that can act by watching and touching things sometimes struggle to fix mistakes during tasks. The paper presents ForceRFT, a method that helps robots learn better corrections by combining human hints and their own experience. It uses force feedback from the robot’s wrist to adjust movements on the fly and improve success in tasks like plugging in a device or wiping a board. This approach led to better autonomous task completion than previous methods relying just on demonstrations or imitation.
What this means in practice
- •For industrial robot operators: Improve robot recovery and precision in assembly tasks by refining actions using force-sensor feedback and human corrections.
- •For robotics engineers: Build adaptable robot control systems that learn from human input and autonomous outcomes to handle contact-sensitive tasks better.
Authors
Yichen Wang, Chaoyang Zhang, Xuqi Su, Jun Ma, Haiyue Zhu, Xiaocong Li
Abstract
Force-conditioned vision-language-action (VLA) policies can respond to contact, but when trained solely on demonstrations, their recovery behavior may be limited by demonstration coverage, and they do not learn from deployment outcomes. Human corrective imitation provides additional recovery examples, but its objective matches local action targets without explicitly optimizing task return. We present ForceRFT, a force-guided residual reinforcement learning framework that learns contact-dependent corrections from human supervision and autonomous task outcomes. A frozen, demonstration-trained SmolVLA-based prior generates force-conditioned action chunks, while a lightweight residual actor refines individual end-effector pose commands using wrist feedback acquired during chunk execution. The decision-time wrist wrench, its temporal change, and the selected base motion condition both residual correction and value estimation. Human corrections supervise the residual actor, while verified autonomous transitions train the twin critics and support value-guided updates to the same actor. Bootstrapping is restricted to autonomous segments, preventing TD credit from crossing human-intervention boundaries. Real-robot experiments on plug insertion, ring-on-peg assembly, and whiteboard wiping show higher autonomous success rates than the evaluated demonstration-trained and residual-imitation baselines. Comparisons with residual imitation support value-guided residual optimization, while plug-insertion ablations indicate the benefit of direct execution-time wrist feedback.