Embodied agents improve goal interpretation with generate-then-rank approach
Generate-then-Rank: Learning Symbolic Goal Interpretation for Embodied Agents
Artificial IntelligenceComputation and LanguageRobotics
Summary
Translating spoken instructions into precise commands that a robot can follow is tricky because mistakes cause the whole task to fail. The authors created a two-step method where a language model first comes up with several possible commands, then a second step picks the best one. They tested this on household robot tasks and found it worked better than simpler methods and those relying on knowledge graphs. This helps robots understand instructions more accurately and act accordingly.
What this means in practice
- •For robotic system developers: Improve robots’ ability to understand natural language commands by generating and ranking potential goals before planning actions in home environments.
- •For voice assistant engineers: Enhance interpretation of user instructions into formal goals that interface with automated systems requiring precise symbolic inputs.
Authors
Jisen Lei, Huizhi Liang
Abstract
Goal interpretation is to translate a natural language instruction into a formal symbolic goal that a planner can execute. It is critical for embodied agents because any error here propagates to every downstream planning and action step. Doing it well requires both generating suitable candidates and reliably selecting among them. We present a two-stage generate-then-rank approach to Goal Interpretation in the Embodied Agent Interface (EAI). First, supervised fine-tuning adapts a Large Language Model (LLM) to EAI's output convention. Second, Candidate-Output Ranking from Errors (CORE) learns preferences over complete outputs from the model's own generated alternatives. We conducted experiments on the household scenes of the Embodied Agent Interface Benchmark. Multiple open source LLMs including DeepSeek-V3 chat, DeepSeek-Reasoner, and Qwen2.5-7B and Qwen2.5-72B were evaluated. The proposed method outperformed the baseline models including LLMs with simple prompting, chain-of-thought, and knowledge graph based approaches.