WARL: Wrench-Augmented Reinforcement Learning for Task-Agnostic Learning in Legged Robots
2026-07-27 • Robotics
Robotics
AI summaryⓘ
The authors propose a new method called Wrench-Augmented Reinforcement Learning (WARL) to help legged robots learn better movements. Instead of only controlling joint angles, their method adds forces and torques (wrenches) to the robot's actions early on to explore more possibilities. They gradually reduce wrench use so the robot eventually relies only on joint control. Experiments showed this approach helps robots learn to move well over different terrains without complex reward settings. However, the authors also found that relying too much on wrenches can lead to unnatural behaviors, pointing to the need for careful design that matches the robot's body.
Reinforcement LearningLegged RobotsAction SpaceWrench (Force and Torque)ExplorationCurriculum LearningQuadruped RobotMotor ControlAblation StudyPhysical Embodiment
Authors
Keita Yoneda, Kento Kawaharazuka, Kei Okada
Abstract
While reinforcement learning for legged robots has achieved high motor performance, it has been constrained by the limited exploration capability of actions confined to the joint space. To address this issue, this study proposes a new method, Wrench-Augmented Reinforcement Learning (WARL), which introduces a wrenche (force and torque) into the action space. The proposed method combines wrench-guided exploration with a success rate-based curriculum mechanism to expand exploration capabilities in the early stages of learning, with the ultimate goal of acquiring behaviors based solely on joint control. Experiments using a quadruped robot demonstrated that WARL can learn robustly across diverse terrains and motor tasks without requiring terrain-specific reward adjustments or complex curriculum designs. Furthermore, an ablation study verified the effectiveness of the Switching Curriculum, which gradually eliminates the wrench. On the other hand, we also show that introducing a wrench can encourage behaviors that do not sufficiently exploit the robot's physical embodiment. These findings suggest that while wrench-based exploration enhancement is effective for improving learning efficiency, designing it in a way that is consistent with the robot's physical structure is a critical future challenge.