Modular world models improve task knowledge reuse in robots
Benchmarking World Models for Continual Learning on Compositional Tasks
Machine LearningRobotics
Summary
It can be hard for robots to learn new tasks without forgetting old ones, especially when tasks share common parts. The authors created a way to test robots that learn by combining parts of old and new tasks in robot hand movements. They found that world models with separate reusable parts handle learning better than usual methods, but still struggle to completely avoid forgetting. This shows there’s room to make smarter robot brains that keep old knowledge while learning new things.
What this means in practice
- •For robotics development teams: Improve robot control systems to better reuse learned task parts and reduce forgetting during ongoing training on new robot manipulation tasks.
- •For industrial automation engineers: Develop adaptive automation solutions that continually learn new manipulations without losing efficiency from past task knowledge.
Authors
Haoyu Zhou, Joe Watson, Anson Lei, Ingmar Posner
Abstract
A desirable property of a world model is the ability to learn continually across tasks, adapting to new environments without forgetting what the agent has already learnt. In particular, the ability to retain and reuse knowledge obtained from prior experiences underpins an agent's ability to efficiently adapt to novel environments, as the dynamics of the physical world can often be described in recurring mechanisms. However, the world model's measure of adaptation entangles two abilities: the speed and capacity to learn unseen tasks, and the reuse of knowledge already acquired, since incoming tasks carry novel content alongside what recurs. In order to isolate knowledge reuse from prior experiences, we propose a compositional continual learning benchmark for world models in robot manipulation. Specifically, we design each task curriculum with compositional tasks that combine aspects of the tasks seen in the sequence. We further factorise this composition along the axes of action and perception to better understand how different input modalities bottleneck knowledge reuse. We evaluate state-of-the-art world models under canonical continual learning methods, alongside a modular world model whose dynamics backbone contains explicitly reusable components. Results show that modularity balances reuse against forgetting better than conventional methods, but none solve the problem fully, leaving clear room for continual world models built to reuse without forgetting. More details are available on our project website: https://object814.github.io/Compositional-Continual-Learning/.