MarkovSim keeps simulation context small for long tasks
MarkovSim: an LLM Agent Harness with Bounded Context for Long-Horizon Computational Simulation
Computational Engineering, Finance, and ScienceArtificial Intelligence
Summary
Simulating complex processes can take a long time and generate too much information for today’s language models to understand at once. The researchers created MarkovSim, a system that carefully manages what the model remembers by summarizing progress at key steps. This lets the model handle long and detailed simulations without getting overwhelmed. Tests on engineering problems showed MarkovSim fixed its own mistakes and only needed human input for tricky judgment calls.
What this means in practice
- •For engineering simulation teams: Run long and complex engineering simulations using LLM agents without exceeding memory limits by summarizing progress at defined states.
- •For software developers for scientific tools: Build simulation software that integrates bounded context management to maintain efficient LLM reasoning over extended computations.
Authors
Yeui-Lung Lei, Wen-Bo Chen, Pan Zhang
Abstract
Recent studies have applied large language model (LLM) agent systems to computational simulation workflows. Such workflows are long-horizon, so the information generated can exceed what an LLM can reason over at once, its context window. Context management is therefore essential, yet existing systems have either neglected it or treated it as a supporting mechanism rather than a subject of study. This paper presents MarkovSim, an LLM agent harness for long-horizon computational simulation, whose core is a context management scheme built into the workflow. The workflow record is written to a memory system with a predefined schema, and at workflow-defined boundaries, namely the state transitions of a finite-state machine a compact representation of the solution progress is constructed from it as the agent's context. Instantiated on finite element analysis and evaluated on six benchmark problems, MarkovSim passed 16 of 18 runs and corrected its own faults in both the solver operations and the model physics. The engineer was consulted only where engineering judgement was needed. The context remained bounded throughout every run, regardless of its length.