Reward-Free Continual Adaptation for Resilient Space Robots
2026-08-24 • Robotics
RoboticsArtificial IntelligenceMachine Learning
AI summaryⓘ
The authors study how space robots can keep working well even when their hardware gets damaged, which makes normal control methods fail. They point out that learning methods usually need clear reward signals to improve, but in space, these rewards are hard to measure. To solve this, the authors create a way for robots to learn without new rewards by using a model trained beforehand to predict rewards internally. When the robot is used in a difficult setting, it updates its understanding of the environment without real rewards and fine-tunes its behavior using imagined experiences. They tested this on various space-related tasks where the robot's parts were severely damaged and the approach helped the robots adapt.
space roboticshardware degradationcontinual reinforcement learningreward signallatent-state world modelmodel-based agentunsupervised rolloutspolicy trainingsimulated environmentmorphological failures
Authors
Andrej Orsula, Miguel Olivares-Mendez, Carol Martinez
Abstract
Space robots operate in extreme environments where hardware degradation can critically compromise traditional control strategies. While continual reinforcement learning offers a promising mechanism for online adaptation, it inherently requires access to a reward signal during deployment. However, precise reward computation in space is often infeasible due to the lack of external tracking systems and the overall complexity of the environment. To address the challenge of unobservable rewards, we introduce a reward-free continual learning framework that leverages latent-state world models. By pre-training a model-based agent across diverse simulations, the world model learns a robust predictor of the reward structure within its latent space. Upon deployment to an environment with severe hardware degradation, we freeze the observation encoder and reward predictor to update only the transition dynamics of the world model through unsupervised rollouts. By training the policy entirely on imagined trajectories generated by this updated world model, the agent adapts to altered dynamics without receiving new rewards. We demonstrate our approach across simulated planetary traversal, orbital navigation, and precision assembly tasks subjected to severe morphological failures.