PRACTICE: From Experience to Expertise in Self-Evolving Embodied Agents
2026-08-31 • Machine Learning
Machine Learning
AI summaryⓘ
The authors present PRACTICE, a method to help robots or AI agents improve their skills over time by learning from past experiences without changing their main task-solving system. Instead of relying on fixed rules, their system learns how to update a skill library by adding, refining, or removing skills based on past successes and failures. They train this skill learner in two stages, starting from perfect examples and then learning from real varied experiences. Experiments show that PRACTICE consistently improves performance and does better than previous methods on certain robotics benchmarks.
multimodal large language modelsembodied agentsskill learnerexperience trajectoriesskill librarytask executoronline distillationEB-ALFREDEB-Habitat
Authors
Ziyi Bai, Siqi Li, Tinglei Huang, Börje F. Karlsson
Abstract
Recent studies have shown that multimodal large language models (MLLMs) can serve as embodied agents, translating language instructions and visual observations into executable plans. However, building agents that can continually improve through interaction and rapidly adapt to their environments remains challenging. Summing up experience from past interaction trajectories provides a promising solution, but existing experience-based methods often rely on manually designed prompting workflows to extract and update skills. Such fixed procedures may struggle to learn updated skills from new and diverse experiences. We introduce PRACTICE, which trains a skill learner to discover and maintain a persistent skill library from past interaction trajectories while keeping the task executor frozen. Given the historical accumulated skills and incoming trajectories, the skill learner produces structured batch-edits that add, refine, merge, or remove skills, and then hierarchical consolidate all collected edits into a consistent updated skill library. We train the learner with a two-stage curriculum. First, it learns basic skill generation and library maintenance from oracle trajectories. Then, by contrasting successful and failed trajectories from heterogeneous executors on the same tasks, it learn to identify invalid action patterns and recovery strategies. Finally, we apply online skill-edit distillation to align the skill learner with a stronger teacher on its current edit distribution to further improves the policy. Experiments demonstrate that a compact skill learner delivers consistent performance improvements across successive library-update rounds for multiple frozen executors. On EB-ALFRED and EB-Habitat, PRACTICE further outperforms the strongest experience-based baselines. Project resources are publicly available at: https://baai-agents.github.io/PRACTICE