Neuro soft symbolic framework improves robotic task planning from perception
A Fully Differentiable Neuro-Soft-Symbolic Framework for Perceptual Task Planning
Artificial IntelligenceRobotics
Summary
Planning a set of actions based on what a robot sees is hard because scenes are uncertain and rules must be followed. The authors created a method that combines seeing and planning in one smooth system, allowing uncertainties to be handled better and improving results. Their approach refines what the robot perceives during planning, leading to higher success rates. In tests with block stacking tasks, this system did better and faster than others and worked well with real robotic actions.
What this means in practice
- •For robotics engineers: Enable robots to improve perception and task plans simultaneously for more reliable manipulation tasks.
- •For automation system developers: Develop integrated planning systems that handle uncertain visual input and logical rules in manufacturing or logistics.
Authors
Hongyan Wei, Wael AbdAlmageed
Abstract
Perceptual planning tasks require two key capabilities: accurately perceiving uncertain scenes and planning valid action sequences following logical rules. Conventional methods convert perception into discrete symbolic facts and then plan, discarding perceptual uncertainty and severing task-level feedback to perception. We introduce a generic, fully differentiable neuro-soft-symbolic framework that connects visual perception and task planning within a single computational graph. The framework maintains a continuous soft symbolic state, lifts domain rules into a differentiable soft-$T_P$ transition operator, and optimizes action logits over a short planning horizon. Gradients from the planning objective can also update the perception parameters, allowing task-relevant perceptual representations to be refined during planning. On Blocksworld, our method solves 40/40 LatPlan-40 tasks and 596/600 PlanBench-600 tasks, compared with 33/40 for LatPlan and 587/600 for the reasoning-model baseline, while requiring substantially less computation and time. In the perceptual-uncertainty ablation, our method improves the success rate from 59\% with frozen perception to 83\%. We further conduct task-and-motion simulations on Blocksworld scenes, providing an execution-level validation of the compatibility between decoded task plans and downstream robotic motion execution.