LLM planners balance safety and efficiency under uncertainty
Planning Commitment of LLM under Partial Observability
Artificial IntelligenceComputation and Language
Summary
Planning with large language models (LLMs) in situations where not everything is known can be tricky. The authors show that some LLMs try to commit to a full plan quickly, which can lead to costly mistakes if assumptions are wrong, while others play it safe by only planning based on what they know, which uses more computing calls. Both strategies perform similarly overall, but each has clear trade-offs between making fewer errors and using fewer resources. They also find that LLMs often treat background knowledge as absolutely true, which can be risky if this knowledge is unreliable. The authors suggest adjustable strategies to balance safety and efficiency.
What this means in practice
- •For autonomous robot developers: Design robots that better manage planning risks when operating in uncertain environments using adjustable planning strategies balancing safety and efficiency.
- •For software engineers building ai assistants: Improve AI assistant planning reliability by tuning the degree of plan commitment in uncertain user scenarios to reduce errors or API usage cost.
Authors
Cassie Huang, Zhanwei Cao, Yeawon Lee, Nathalia Gomez, Zhuoyi Yang, Ian G. Harris, Tiffany Do, Jiayuan Mao, Li Zhang
Abstract
LLM-based planning agents commit to plans of different lengths before interacting with a partially observable environment. Some aggressively commit to a long plan based on assumptions about unobserved states, while others conservatively plan only over what they have observed. Both behaviors emerge implicitly in pre-trained models, whether the LLM outputs a plan directly or generates a PDDL formulation for a classical solver. Aggressive commitment risks costly failures, while conservative commitment risks excessive LLM calls. This choice pits safety against efficiency, a trade-off that has been neither characterized systematically nor made controllable. We investigate aggressive and conservative LLM planners and formalizers on two partially observable domains, where priors are provided as either explicit statistics or common-sense knowledge. Across three LLMs, both strategies reach similar accuracy. Conservative methods achieve fewer failed attempts (safety) at the cost of more API calls (efficiency). Aggressive methods fail more often but replan efficiently only when priors are informative. Interestingly, analysis shows that LLMs treat the provided priors nearly as facts, which risks failure when priors are unreliable. We also explore hybrid strategies with tunable hyperparameters that adjust the balance between safety and efficiency.