PlayTrain enables fast training on JavaScript-generated video games
PlayTrain: An Efficient Reinforcement Learning Framework for LLM-Generated Adaptable JavaScript Games
Machine Learning
Summary
Training AI agents to learn video games usually requires a lot of hand-coded work to build or change game environments. The authors created PlayTrain, which uses large language models to automatically write simple JavaScript games from short instructions and runs them in a standard setup for training AI. This approach speeds up the process and allows AI to be trained on new or modified games efficiently. It also supports creating variations of games to test different learning challenges.
What this means in practice
- •For reinforcement learning engineers: Train AI agents efficiently on customized or newly generated JavaScript games without manual environment coding.
- •For game developers: Rapidly prototype and test new game mechanics or levels by generating JavaScript games using language models and training agents on them.
Authors
Ryan Truong, Lance Ying, Samuel J. Gershman, Kazuki Irie
Abstract
While many video-game environments (VGEs) have played crucial roles in advancing reinforcement learning (RL), developing novel VGEs or modifying existing ones to support new features, has been a laborious process requiring extensive hand-coding. Here we present PlayTrain, an RL framework that combines the abilities of large language models (LLMs) to robustly generate JavaScript (JS) games from a minimal human prompt, and an efficient pipeline that can run any JS game in a standard 'gym' environment. Not only are recent LLMs particularly good at writing JS code, but the JS format also allows users to easily play generated VGEs, while PlayTrain enables us to train RL agents on the exact same games. We demonstrate multiple use cases of PlayTrain, including cloning well-known Atari and ProcGen games in simple JS, where PlayTrain trains pixel-based agents end-to-end at over 1M agent-decisions per second on a single GPU node; and creating modified versions thereof (e.g., that support novel test sets, procedural generation logics, or game dynamics). Through PlayTrain, we reimagine RL VGE development: all we need is a single JS file, generated and modified through an LLM. We discuss promising future RL research directions that PlayTrain unlocks.