ProbeAct: Probe-Guided Training-Free Failure Recovery in Vision-Language-Action Models
2026-06-08 • Robotics
Robotics
AI summaryⓘ
The authors present PROBEACT, a system that helps robot models trained to follow language commands handle mistakes without retraining. PROBEACT watches the robot's internal signals and guesses where key objects are, detecting when grasping or placing actions fail. It then gently adjusts the robot's movements to avoid repeating these failures, acting like a safety net during task execution. Tested on a standard benchmark, PROBEACT improved robot success rates without changing the original model.
Vision-Language-Action modelsrobotic manipulationruntime intervention3D object position estimationkinematic state machineControl Barrier Functiongrasp failure detectionpolicy generalizationmulti-object trackingbenchmark evaluation
Authors
Fan Zhang, Seongbin Park, Baharan Mirzasoleiman, Shariar Talebi, Nader Sehatbakhsh
Abstract
Vision-Language-Action (VLA) models demonstrate strong perfor-1 mance on language-conditioned robotic manipulation within their training dis-2 tribution, yet their generalization capabilities remain fundamentally limited. They3 lack the robustness required to handle perturbations, frequently failing when con-4 fronted with lighting changes, altered camera viewpoints, or small initial-state5 variations. We propose PROBEACT, a training-free runtime intervention frame-6 work that detects and recovers from grasping and placement failures in pre-7 trained VLA policies without modifying their weights or requiring additional8 demonstrations. PROBEACT combines three components: (i) a lightweight multi-9 target hidden-state probe that predicts the 3D positions of task-relevant objects10 from intermediate VLA features, with Hungarian-matched identity tracking for11 multi-object scenes; (ii) an object-agnostic kinematic state machine that detects12 grasp, transport, and placement failures using only gripper-internal signals and13 end-effector kinematics; and (iii) a hierarchical Control Barrier Function (CBF)14 filter that encodes repeated-failure locations as soft safe-set constraints, mini-15 mally correcting VLA actions while preserving baseline behavior. As a plug-and-16 play, training-free intervention loop, PROBEACT is orthogonal to existing train-17 ing pipelines. Evaluated on the LIBERO-plus benchmark, our framework acts as18 a universal safety net, improving the success rate of the OpenVLA-OFT model19 from 69.6% to 74.1%, while demonstrating broad applicability to both base and20 fine-tuned VLA policies.