SafeRun: Enabling Determinism in LLM Planning for Running
2026-06-08 • Computation and Language
Computation and LanguageArtificial Intelligence
AI summaryⓘ
The authors address the problem that large language models (LLMs) can plan using natural language but sometimes make unsafe or unreliable decisions, especially when strict safety rules must be followed. They created SafeRun, a method that combines an LLM's flexible understanding with a separate system that enforces hard safety rules with full certainty. The authors tested SafeRun on running plans that must respect real physical and safety limits, finding it always followed safety rules better than existing methods while still understanding instructions well. They also provide a public benchmark for others to test running plans under safety constraints.
Large Language ModelsNatural Language PlanningDeterministic SolverSafety ConstraintsDecoupled ArchitectureRunning PlanningProbabilistic ModelsInstruction FollowingBenchmark DatasetCodeAct
Authors
Meilin Chen, Zepeng Zhai, Jiaxuan Zhao, Yuan Lu
Abstract
Large Language Models enable flexible natural-language planning but remain unreliable in determinism-critical domains due to their probabilistic nature. This limitation is especially problematic in running planning, where violating safety rules can lead to safety risks. We propose SafeRun, a framework for deterministic LLM-based planning via a decoupled architecture. SafeRun separates soft interpretation by an LLM from hard constraint enforcement by a deterministic solver, ensuring strict safety constraints while preserving natural-language flexibility. To validate SafeRun, we build a comprehensive benchmark for running planning under realistic physiological and safety constraints. Experiments across five LLMs show that SafeRun achieves 100\% safety score (vs.\ 79.1\% PE average and 97.6\% CodeAct average) while maintaining competitive instruction-following scores. The SafeRun benchmark is publicly available at \href{https://huggingface.co/datasets/zzp-seeker/SafeRun-RunPlanning-Benchmark}{huggingface}.