Robot grasping improved by steering feasible grasp options at inference
Steer2Grasp: Inference-Time Embodiment-Aware Steering for Diverse Physically Feasible Grasp Diffusion
Robotics
Summary
Robots need to pick up objects without bumping into things or reaching awkwardly, but current methods often suggest grasps that don’t work physically. The authors present a way to adjust grasp suggestions on the fly without retraining, guiding the robot’s hands to more feasible positions considering its shape and surroundings. Their method uses a clever technique to shift grasp suggestions toward better options while keeping the original knowledge intact. This approach works for both single and dual-arm robots, improving their ability to plan realistic grasps for a variety of objects and environments.
What this means in practice
- •For robotics engineers: Improve robotic arm grasp planning by steering grasp candidates to feasible poses without retraining models.
- •For automation system integrators: Enhance dual-arm robot grasping in cluttered or constrained environments by refining grasp suggestions during deployment.
Authors
Vignesh Vembar, Ayush Kaura, A Padmaprabhan, Siddharth Sinha, Kailash Nagarajan, Keshab Patra, Md Faizal Karim, K Madhava Krishna
Abstract
Current grasp diffusion models provide rich priors for generation, yet their object-centric approach can violate the kinematic and collision constraints imposed by the embodiment and the environment. Existing embodiment-aware methods primarily perform local corrections around generated grasps through gradient guidance or optimization, making it difficult to recover from fundamentally infeasible modes. We present Steer2Grasp, a training-free, embodiment-agnostic framework for inference-time grasp steering that adapts a frozen Cartesian grasp diffusion model using deployment-specific rewards. Through Feynman-Kac (FK) inspired particle reweighting and resampling, the method reallocates population mass from infeasible to high-reward grasp modes, enabling population-level mode transitions without modifying the pretrained diffusion model or requiring differentiable constraints. The framework enables a unified treatment for single and dual arm grasping through reachability and collision aware rewards, followed by gradient free gripper level local refinement. Across diverse objects, robot embodiments, and constrained environments, our method substantially improves feasible grasp generation while maintaining proximity to the underlying grasp prior.