Humanoid locomotion improved by separating aesthetics from physical smoothness

PhySA: Decoupling Aesthetic and Physical Rewards for Humanoid Locomotion

Robotics

Summary

Humanoid robots often move jerkily when trained without care for smoothness, which can wear out their joints. This paper finds that AI models judging movement aesthetics tend to prefer less smooth, high-energy motions. The authors introduce PhySA, a method that separates aesthetic rewards from physical smoothness penalties, enabling smoother robot walking. Their approach reduces joint jerkiness significantly in simulation, although results vary across different robot types.

What this means in practice

  • For robotics engineers: Develop smoother walking policies for simulated humanoid robots by decoupling aesthetic and physical reward signals during training.
  • For robot control developers: Improve actuator lifespan of humanoid robots by incorporating phase-triggered correction channels that adjust physical penalties separately from visual quality rewards.

Tested on simulated data.

Authors

Yunwan Fu, Sicheng He, Jiageng Mao, Yue Wang

Abstract

Humanoid locomotion policies trained without an explicit smoothness objective tend to produce jerky, high-energy gaits that stress actuators. Vision-language models (VLMs) offer an appealing reward source: given a rendered rollout, a VLM returns an aesthetic quality score on [0,10] without task-specific feature engineering. We find, however, that aesthetic preference and physical smoothness point in opposite directions: across four production VLMs, clips with higher joint acceleration receive higher aesthetic scores (mean Pearson r=+0.66). We study PhySA (Physical Smoothness and Aesthetics), which keeps the two on separate, independently removable reward channels: a sparse, asynchronous VLM score and a phase-triggered correction channel that adjusts physical penalty weights from simulator state. Ablations attribute the smoothness gain to the correction channel. On MuJoCo Humanoid-v5 at 2.6M steps, PhySA lowers joint acceleration by 26 to 32% relative to vanilla PPO, Text2Reward, and Eureka (three seeds), at the cost of shorter episodes. In a single extended run it lowers joint acceleration by 59.5% relative to Prompt2Policy, while a three-seed replication shows a smaller, non-significant reduction (7.2%). The gain does not transfer to IsaacLab or the Unitree G1, even after rescaling the smoothness term.