Information-Based Exploration via Random Features for Reinforcement Learning
2026-07-20 • Machine Learning
Machine Learning
AI summaryⓘ
The authors developed a new method called Random Feature Information Gain (RFIG) to help computers explore and learn better in complex environments, like video games or robot navigation. Unlike other methods that rely on complicated neural networks and hard-to-understand uncertainty estimates, RFIG uses a mathematical trick with random features to measure how much new information the computer can gain. This approach simplifies calculations and comes with guarantees on accuracy. The authors show that RFIG works well in tests and can easily be added to existing learning algorithms.
Reinforcement LearningExploration StrategiesRandom Fourier FeaturesInformation GainBayesian Kernel MethodsDeep LearningUncertainty EstimationOptimism-based ExplorationRepresentation LearningControl Tasks
Authors
Waris Radji, Odalric-Ambrym Maillard
Abstract
Representation learning has enabled classical exploration strategies to be extended to deep Reinforcement Learning (RL), but often makes algorithms more complex and theoretical guarantees harder to establish. We introduce Random Feature Information Gain (RFIG), grounded in Bayesian kernel methods theory, which uses random Fourier features to approximate information gain and compute exploration bonuses in non-countable spaces. We provide error bounds on information gain approximation and avoid the black-box aspects of neural network-based uncertainty estimation, for optimism-based exploration. We present practical details that make RFIG scalable to deep RL scenarios, enabling smooth integration into standard deep RL algorithms. Experimental evaluation across diverse control and navigation tasks demonstrates that RFIG achieves competitive performance with well-established deep exploration methods while offering superior theoretical interpretation.