Papers for

legged robot developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Actuator dynamics curriculum improves learning in tough legged robot tasks

Actuator Dynamics Curricula for Narrow-Viability Tasks in Legged Robot Learning

Abstract: Reinforcement learning has produced capable controllers across a broad range of legged-robot tasks, but a subset of these tasks fail to converge under standard training: those for which most exploration trajectories terminate before producing useful gradient signal. To address such tasks we introduce the \emph{Actuator Dynamics Curriculum}, a procedure that initializes joint stiffness at a high value and anneals it toward the system-identified value as completed episode lengths grow. Using a cart-pole system as a representative example, we show that higher closed-loop joint natural frequency under critical damping enlarges the viability kernel of the underlying Markov Decision Process, increasing the fraction of initial states from which the task is feasible. We validate the kernel monotonicity on the cart-pole and apply the curriculum to a quadrupedal-to-handstand transition on the Boston Dynamics Spot, a narrow-viability task where training under fixed identified stiffness plateaus at a policy that never completes the transition. The trained policy executes the transition in simulation across 10 seeds and transfers to hardware. More broadly, our results suggest that simulated actuator dynamics is a useful axis along which to design curricula for tasks in which exploration is bottlenecked by termination conditions rather than by reward signal.

Tue 8 SeptRobotics
The gist
Some robot tasks are so tricky that usual training methods fail to teach robots how to complete them. The authors made a new training method that starts with robot joints being very stiff and then gradually moves to normal stiffness as the robot improves. This helps the robot try out more ways to move without failing right away. They tested it on a robot doing a hard move, a quadruped flipping into a handstand, and showed it works better than normal training, even on real hardware.
Open 2609.09492v1