Risk-aware system improves robot trust in ambiguous instructions
Risk-Aware Semantic Grounding for Trustworthy LLM-Based Robot Planning
RoboticsArtificial Intelligence
Summary
Robots using large language models to plan their movements can sometimes get confused by unclear or conflicting instructions. The authors introduce a new way for these robots to recognize when commands are risky to follow—like when instructions are vague or don't match the environment. This system helps robots decide whether to carry out a task, ask for clarification, or refuse to act, making robot behavior more reliable and safe. They tested their approach on a new set of navigation tasks and found it better at spotting problematic instructions than previous methods.
What this means in practice
- •For robotic system developers: Incorporate risk-aware instruction validation to improve robot navigation safety and reliability in uncertain environments.
- •For autonomous vehicle engineers: Enhance decision-making modules to identify and reject ambiguous or conflicting navigation commands before execution.
Authors
Łukasz Sobczak, Nur Keleşoğlu, Sławomir Piotr Nowak
Abstract
Large language models (LLMs) are increasingly used as high-level planners in robot navigation, but their outputs may become unreliable when instructions are ambiguous, unsupported by the environment, or semantically inconsistent. This paper presents a Risk-Aware Semantic Grounding framework for trustworthy LLM-based robot planning. Unlike existing LLM-based planners that primarily optimize plan generation, we formulate semantic grounding reliability as a multi-dimensional risk estimation problem. The proposed architecture explicitly models grounding uncertainty through ambiguity, hallucination and semantic-conflict risks before planning occurs, enabling the system to decide whether to execute the instruction, request clarification, or reject it. To evaluate the approach, we introduce TRUST-NAV, a benchmark containing both standard navigation tasks and risk-inducing instruction scenarios. Experimental results show that while conventional LLM planners achieve strong performance on valid navigation tasks, the proposed framework substantially improves ambiguity detection and semantic conflict rejection. These findings suggest that trustworthy robot planning should be evaluated not only by task completion, but also by the ability to recognize when execution should not occur.