Minimal Local Simulation Foundations for LLM- and VLM-Driven Agents in 2D and 3D Environments
2026-08-24 • Multiagent Systems
Multiagent SystemsArtificial Intelligence
AI summaryⓘ
The authors created two simple simulation tools that let computers run and test virtual agents using language models on regular computers. One tool works in 2D where agents can move around, talk, and react to things like fire nearby. The other works in 3D where an agent sees images and gives movement instructions using natural language. Both are open for users to change and learn from, making them good starting points for studying how language models can control virtual agents.
large language modelsvision-language modelsmulti-agent simulationdigital twinnatural language processing2D environment3D environmentagent communicationprototypeopen source
Authors
Ryuki Hyodo
Abstract
Large language models (LLMs) and vision-language models (VLMs) are expanding the range of behaviors that can be represented in agent-based simulations, but many contemporary platforms are difficult to study, modify, or run on ordinary computers. We present two intentionally minimal simulation foundations for education and rapid prototyping. SD-AgentFoundry-2D provides a two-dimensional multi-agent environment in which locally hosted LLM agents move, communicate, respond to place occupancy, and encounter spatially localized fire events. SD-AgentFoundry-3D provides a three-dimensional digital-twin environment in which a locally hosted VLM receives first-person images and produces natural-language movement instructions. Both codebases are designed to run locally on macOS, Windows, and Linux and are deliberately left open to modification rather than developed as finished applications. Together, they offer accessible starting points for learning about generative social simulation and for building domain-specific extensions.