Learned controllers improve recovery while preserving past successes
Sequential Recovery Updates for Learned Controllers under Successful-Trajectory Preservation
RoboticsArtificial Intelligence
Summary
When robots or systems learn new ways to fix problems, they often forget how to handle things they previously did well. The authors came up with a way to add new recovery skills without messing up what already works. Their method tests new fixes carefully, only accepting those that don’t interfere with successful past actions. They showed this works well on humanoid robot simulations, adding many recoveries without losing earlier abilities.
What this means in practice
- •For robotics developers: Add new recovery behaviors to robot controllers while ensuring all previously learned successful motions remain unchanged.
- •For autonomous vehicle engineers: Improve adaptive driving controllers to handle new disturbances without losing ability to follow previously successful routes.
Authors
Zongtan Li, Tianming Liu, Ye Zhao
Abstract
A competent learned controller already recovers from many disturbances. Improving it means learning a correction, and learning when not to apply it. Freezing a skill's parameters does not settle the second half. In a controller that routes among frozen skills, a new entry rule can redirect the states an old skill succeeded on. We therefore constrain entry instead of parameters. Each update searches for a feedback skill at a measured failure. It fits an entry rule that reads observations and rejects every state on an accumulated successful trajectory. The skill is admitted only if the composed controller reproduces the new recovery on its own and replays every protected trajectory exactly. A finite-trajectory condition states when this preserves the previous controller. One mechanism runs on four streams: two motions on each of two humanoid systems. Two updates per stream add 62 recoveries over 192 withheld perturbations, and lose none of the base controller's successes. We then price that guarantee. Removing the constraint recovers 33 more conditions but stops eight protected ones from completing at all, and excluding only what admission proves necessary recovers none of that cost.