Grasp feasibility predictors fail when obstacles change approach paths
Auditing Grasp Feasibility Predictors with Planner-Verified Counterfactuals
Robotics
Summary
Robots use grasp feasibility predictors to decide if a hand position can safely pick up an object. This paper shows that these predictors often ignore important obstacles by testing with new, inserted obstacles and checking with motion planners. The authors find that predictor accuracy drops to chance when obstacles block the grasp path, meaning the models don't truly understand collision risks. They propose a new testing method and offer improved predictor designs that better consider obstacle clearance.
What this means in practice
- •For robotics engineers: Identify and improve grasp predictors by verifying if they truly account for obstacles in robot manipulation tasks.
- •For robot software developers: Use planner-verified counterfactual tests to audit and refine grasp prediction modules in robotic grasping systems.
Authors
Sajid Ansari, Antony Thomas, Girish Varma
Abstract
Grasp-feasibility predictors report near-perfect accuracy on Task and Motion Planning benchmarks, but high accuracy on these benchmarks does not verify that a predictor uses the collision geometry that the motion planner checks. We propose a planner-verified counterfactual (CF) test: holding the grasp pose fixed, we insert a solid obstacle into the approach corridor, add its points to the model input, and let a planner re-derive the label, which flips from feasible to infeasible. Across eight predictors spanning point clouds, voxels, depth, and object tokens, CF-pair AUROC falls from high natural AUROC (0.920-0.996) to chance (0.470-0.504), and the flip holds across four planners, two arms, and a real-object obstacle. Predictors fail at different resolutions, from a bounding box to a 12.5 cm voxel. In the point-cloud models we dissect, the obstacle either moves the representation in a direction unrelated to whether it blocks the grasp or is not registered at all. A clearance-aware read of the corridor points, trained on the unmodified benchmark with no counterfactual scenes, reaches 0.983 CF-pair AUROC, but it remains bound to its trained obstacle placements; a placement-invariant clearance representation is still open. We release the counterfactual labels, metrics, datasheet, and relabeling pipeline at https://github.com/codewithsajid/graspfc-cf.