Spiking neural network predicts future game states in reinforcement learning
Spiking Neural Network Predicting Sequence of the External Worlds States in Model-Based Reinforcement Learning
Neural and Evolutionary Computing
Summary
Predicting what will happen next in a game or the real world is hard. The authors developed a brain-inspired spiking neural network that can guess the sequence of future states starting from the current one. Instead of building a full model of how the world works, this network links smaller parts that predict the next step and chains those predictions together. They tested this system on a classic video game called Atari ping-pong and showed it could predict game states over time.
What this means in practice
- •For robotic control teams: Design robots that anticipate sequences of future states using spiking neural networks for better model-based decision making.
- •For game ai developers: Implement brain-inspired state prediction modules in game-playing agents to improve lookahead planning and action selection.
Tested on one dataset.
Authors
Mikhail Kiselev
Abstract
This paper presents a spiking neural network (SNN) designed to predict the sequence of the external world states starting from the current world state. This SNN does not create the world dynamics model - instead it incorporates the SNN trained to predict the next world state and provides all mechanisms necessary to make the chain of predicted world states. These mechanisms are entirely spiking - they are implemented as spiking neuron ensembles. The present article describes this neuronal structure and tests its operation on a classic RL benchmark - ATARI ping-pong.