Embodied agents learn from parts of past tasks to improve actions
EmbodiExp: Compositional Experience Transfer for Embodied Agents
RoboticsArtificial IntelligenceComputer Vision and Pattern RecognitionMachine Learning
Summary
Robots or digital agents often struggle to reuse what they learned from one task when doing another task. The authors introduced EmbodiExp, a method that breaks tasks into smaller goals that can be checked individually. This helps agents pick and combine these smaller experiences during new tasks, even if some attempts previously failed. Their tests showed this approach helps agents perform better on several benchmark problems.
What this means in practice
- •For robotics developers: Improve robots' ability to reuse parts of earlier skills when completing varied tasks by composing verified subgoals during execution.
- •For game ai programmers: Enable game characters to adapt previously learned action sequences for new objectives by verifying subgoal outcomes and composing experiences online.
Authors
Zhifeng Gu, Ying Chen, Rui Jiang, Lei Yi, Bing Wang
Abstract
Experience can help embodied agents turn high-level plans into grounded actions, but transferring it across tasks remains challenging: previously effective strategies may be inapplicable to the current state or fail to produce their intended outcomes. We present EmbodiExp, a framework for constructing transferable experiences and composing them online during embodied task execution. Our key observation is that tasks often share subgoals whose completion can be assessed independently of overall task success. We therefore introduce an outcome-grounded subgoal experience that specifies a subgoal, applicability conditions, an adaptable policy, and a verifiable outcome. During execution, applicable experiences guide action generation, while verified outcomes can establish the conditions required by subsequent experiences, enabling online composition. Subgoal-level outcome attribution further enables failure utilization: failed trajectories can still contribute verified local successes and condition-specific guidance on actions to avoid. Experiments on multiple embodied benchmarks under a fold-based protocol with frozen experience banks demonstrate the effectiveness of EmbodiExp.