TAMPAgent improves robot planning for complex household tasks
TAMPAgent: Agentic Belief-Space Planning for Mobile Manipulation
Robotics
Summary
Robots working in the real world often don’t know everything about their surroundings, which makes it hard to move and manipulate objects safely. The authors developed TAMPAgent, a system that combines powerful language-based reasoning with detailed motion planning to help robots understand and act better in uncertain environments. This system plans by thinking about what it knows and doesn’t know, making step-by-step decisions and updating its plans as it goes. Tested on two different robot platforms, TAMPAgent performed much better than previous methods in completing complex home tasks.
What this means in practice
- •For robotics engineers: Build robots for household chores that plan actions with uncertainty and update their plans as they interact.
- •For industrial automation teams: Improve mobile robot manipulation in factories where environments are complex and only partially known.
Authors
Utkarsh A. Mishra, Alperen Degirmenci, Yongxin Chen, Danfei Xu, Tucker Hermans, Caelan Garrett
Abstract
Open-world robotic mobile manipulation requires acting in physical environments with only partial information. While recent Large Vision-Language Model (LVLM) agents show strong open-world reasoning capabilities, existing architectures that rely on direct tool calling struggle with geometric uncertainty and thus are limited to fully-observable tabletop environments. In contrast, Task and Motion Planning (TAMP) systems can solve long-horizon tasks under partial observability but incur high computational costs and require closed worlds. We propose \textbf{TAMPAgent}, an agentic planning framework that unifies the reasoning of LVLMs with the structured constraint satisfaction of TAMP systems. We endow an LVLM agent with a belief-space planning model operating over an extensible API of perception, planning, and control tools. The agent formulates a partial task plan composed of parameterized observation and execution actions, which is subsequently completed using classical search algorithms. After extracting the plan's geometric constraints, the agent leverages its tool API to bind valid parameter values, satisfying feasibility requirements and producing an executable motion plan. Upon execution, the agent continuously updates its belief history and repeats this closed-loop process by interleaving observation and manipulation actions. Evaluated across complex household mobile manipulation tasks on Rainbow Y1 and Unitree G1 platforms, TAMPAgent significantly outperforms direct tool-calling agentic baselines, demonstrating that integrating TAMP constraint satisfaction into agentic planning loops is essential for unstructured, partially observable environments.