Continual-RL for Generalization in Autonomous Racing on the RoboRacer Platform
2026-07-27 • Robotics
Robotics
AI summaryⓘ
The authors address the problem of teaching robots to adapt to new environments, especially when simulations can't cover every real-world detail. They focus on continual reinforcement learning, which helps robots keep learning as conditions change. Using the RoboRacer competition as a test, they created a method that trains a general driving policy from real-world data and can quickly adjust to new tracks in just 15 minutes, outperforming traditional controllers. They also compare their approach to offline reinforcement learning and study how flexible these methods are in adapting.
Reinforcement LearningContinual LearningReinforcement Learning in RoboticsSim-to-Real TransferRoboRacer CompetitionContinual BackpropagationOffline Reinforcement LearningPolicy Fine-TuningPlasticity in LearningAutonomous Racing
Authors
Joel Siegert, Edoardo Ghignone, Michele Magno
Abstract
A key challenge in modern robotics is to adapt to changing environments, a challenge that is exacerbated when simulations cannot encompass every possible real-world configuration, and therefore Reinforcement Learning (RL) in the physical world becomes necessary. Continual Reinforcement Learning provides the tools to address this challenge; however, both the frameworks and the methods remain underexplored. Autonomous Racing and in particular the RoboRacer competition provide a testing ground for such methods, as learning to drive on a new track-floor combination with the least amount of new experience naturally frames a continual learning problem. This work tries to address this gap by proposing a continual RL framework based on Continual Backpropagation that is able, with only real-world data, to train a generalistic policy on a set of tracks and then fine- tune it within 15 minutes to outperform classical controllers. Furthermore, a comparison method based on offline RL is proposed, and a simulation analysis of the plasticity properties of the methods is conducted.