Neurosymbolic Embodied Agents
2026-08-17 • Robotics
RoboticsArtificial IntelligenceComputation and Language
AI summaryⓘ
The authors created a smart agent that breaks down complex household tasks into two parts: exploring the environment to understand what’s around and then planning actions using strict rules. First, the agent looks around and uses language and vision models to figure out what objects and goals are present, turning this into a simple symbolic map. Next, it plans step-by-step using a system that only allows feasible actions, checking for the best path with a search method. Their approach works better than other methods on test environments and uses less computing power, with most failures due to mistakes in recognizing the environment rather than planning.
vision-language modelssymbolic planningPDDLMonte Carlo tree searchegocentric observationtransition modelVirtualHomeALFWorldembodied agents
Authors
Mohammad Albinhassan, Yuming Feng, Alessandra Russo, Pranava Madhyastha
Abstract
Language and vision-language models generate plausible embodied plans but do not guarantee executability, as their outputs can violate environment dynamics or act on incorrectly grounded entities. We present a neurosymbolic agent that factors long-horizon household tasks into task-directed visual exploration and constrained symbolic planning. In the first phase, a vision-language model and exploration harness acquire goal-relevant predicates and instance bindings from egocentric observations and grounded interactions, producing a symbolic initial state. In the second, a PDDL transition model restricts decoding to tokens that extend applicable actions. Monte Carlo tree search then evaluates executable continuations using a domain-independent planning heuristic. The resulting plans are executable by construction under the transition model, with transfer to the environment conditioned on correct visual grounding. On VirtualHome and ALFWorld, open 4B-27B models exceed 90% success in both environments, and our smallest agent substantially outperforms a 27B direct visual policy in each. Constraints and search prove complementary rather than interchangeable: in ALFWorld either alone solves under a third of tasks, whereas their combination solves over 95%. The method also uses several times fewer generated tokens than extended thinking and far fewer model-visible images than direct interaction, and residual failures localize to state acquisition rather than plan generation without any specialized training.