PlayTrain enables fast training of AI agents on AI-generated JavaScript games
PlayTrain: An Efficient Reinforcement Learning Framework for LLM-Generated Adaptable JavaScript Games
Machine Learning
Summary
Making video game environments for training AI has been slow and needs lots of coding. The authors designed PlayTrain, which uses large language models to create simple JavaScript games from small instructions. Then, PlayTrain trains AI agents to play those games efficiently using a common interface. This approach lets people quickly generate and modify games to test new ideas in AI learning. It could speed up research by making it easier to build and train on many different games.
reinforcement learninglarge language modelsJavaScriptvideo game environmentsAI game agentsprocedural generationgym environmentpixel-based agents
Authors
Ryan Truong, Lance Ying, Samuel J. Gershman, Kazuki Irie
Abstract
While many video-game environments (VGEs) have played crucial roles in advancing reinforcement learning (RL), developing novel VGEs or modifying existing ones to support new features, has been a laborious process requiring extensive hand-coding. Here we present PlayTrain, an RL framework that combines the abilities of large language models (LLMs) to robustly generate JavaScript (JS) games from a minimal human prompt, and an efficient pipeline that can run any JS game in a standard 'gym' environment. Not only are recent LLMs particularly good at writing JS code, but the JS format also allows users to easily play generated VGEs, while PlayTrain enables us to train RL agents on the exact same games. We demonstrate multiple use cases of PlayTrain, including cloning well-known Atari and ProcGen games in simple JS, where PlayTrain trains pixel-based agents end-to-end at over 1M agent-decisions per second on a single GPU node; and creating modified versions thereof (e.g., that support novel test sets, procedural generation logics, or game dynamics). Through PlayTrain, we reimagine RL VGE development: all we need is a single JS file, generated and modified through an LLM. We discuss promising future RL research directions that PlayTrain unlocks.