SleepWalking: Privileged Representation Shaping for End-to-End Blind Locomotion in Legged Robots
2026-08-31 • Robotics
Robotics
AI summaryⓘ
The authors deal with robots that need to move even when they don’t have full information about their environment. Instead of trying to guess or memorize missing details, they focus on helping the robot's memory keep the important information it learns over time. They created a new method called SWAQ, which trains the robot to remember key physical info by predicting what comes next, but the robot just uses what it remembers when moving. Their approach works better and uses less computing power than previous methods, and their tests show that the robot keeps important info in its decision-making process.
partial observabilityrobot locomotionpolicy learningrecurrent neural networksprivileged informationend-to-end trainingphysical variable reconstructioninference MACssemantic objectiveshistory representation
Authors
Zheng Pan, Tenghui Wang, Peilin Li, Shiyu Zhou, Hao Sun, Yan Ma, Liang Yu, Liang He
Abstract
Partially observable locomotion requires a policy to act when task-relevant properties of the robot--environment state are not fully specified by instantaneous observations. Existing approaches often address this challenge by explicitly estimating missing physical variables or processing extended observation histories through structured architectures. We take a different view: partial observability is fundamentally an information-retention problem. The decisive question is not how task-relevant information enters the network, but whether the policy's internal state retains it. Guided by this perspective, we propose SleepWalking for Robot Locomotion (SWAQ), a one-stage end-to-end framework that uses next-step privileged physical reconstruction to shape what a recurrent history representation retains during policy learning, while the deployed actor uses only a direct history-to-action pathway. Under aligned training settings, SWAQ achieves a 15.0\% higher peak mean terrain level than DWAQ, the strongest non-exteroceptive baseline, while using 44.4\% fewer inference MACs per control step. Layerwise probes further show that information associated with the reconstructed physical variables remains linearly decodable through the policy head up to the layer preceding the action output. Complementary theoretical analysis relates privileged-variable recoverability to the achievable-return gap between history-based and privileged-information policy classes. These results suggest that semantic objectives can structure learning without requiring a corresponding architectural decomposition of the deployed controller.