Effective rank improves robot leg movement from simulation to real world
Mind the Phase: Effective Rank and Representation Health in Legged Locomotion
RoboticsArtificial Intelligence
Summary
Training robots to walk or run using simulation and learning methods can produce jerky or unreliable movements when robots try the behaviors in real life. The authors investigate how neural networks used in these robots represent movements differently during different parts of a step, like when the leg swings versus when it touches the ground. They find certain network features help make these representations clearer and more stable. Using this understanding, they improve training so robots move more smoothly when moving from simulation to the real world.
What this means in practice
- •For robotics engineers: Reduce joint jitter in legged robots by conditioning training on gait phase representations to improve real-world movement smoothness.
- •For autonomous delivery teams: Deploy legged robots with more reliable walking behaviors that transfer smoothly from simulation to actual environments.
Authors
Felipe Tommaselli, Thiago H. Segreto, Juliano D. Negri, Ricardo V. Godoy, Marcelo Becker
Abstract
Reinforcement learning has become the leading paradigm in legged locomotion, enabling complex behaviors from backflips to parkour through massively parallel simulation. Under PPO's non-stationarity, shallow networks remain the de facto architecture, supported by carefully staged curricula and environments, yet the representations these policies learn stay poorly understood, leaving no training-time signal of how they will behave on hardware. In this work, we empirically study locomotion policies through the effective rank of the policy Jacobian and show that conditioning rank on the gait phase exposes architectural structure that global rank averages away. In particular, we find that standard architectural choices, namely layer normalization and residual connections, allocate roughly two more dimensions of effective rank to swing than to stance, which is fully absent in vanilla MLPs. Building on this, we propose a simple recipe that turns these representational signatures into smoother, more reliable sim-to-real transfer. In practice, this results in roughly 3x lower joint jitter that holds from simulation onto a physical Spot, suggesting that representation health is an effective training-time lens to track sim-to-real smoothness.