Vision system improves grasp tool use for prosthetic arms
Extending Affordance Segmentation for Wearable Robots to Multi-Object Images
Computer Vision and Pattern RecognitionRobotics
Summary
Identifying which parts of objects can be grabbed is tricky when many objects appear together. The authors added a laser pointer to the setup to help the system know which object to focus on. They also retrained the neural networks that recognize graspable parts using a method involving realistic mixed reality images. This improved the system's accuracy in picking the correct target by up to 9% without making it slower.
What this means in practice
- •For prosthetic device manufacturers: Integrate laser pointer-guided vision systems to enhance prosthetic grasp accuracy in multi-object environments.$Commercial implications: Enables development of smarter prosthetic arms that better select and grasp objects in cluttered settings.
- •For robotic grasping engineers: Use minor hardware tweaks combined with mixed reality training data to improve object part detection in complex scenes.
Authors
Simone Lugani, Matteo Montani, Paolo Gastaldo, Rodolfo Zunino, Edoardo Ragusa
Abstract
This paper proposes a refinement to existing vision-based portable pipelines for the recognition of graspable object parts. Such pipelines can effectively support automatic grasp selection and preshaping in upper-limb prosthetic devices. However, realistic scenarios in which multiple objects are present still represent a challenge. The proposed method exploits a minor modification to the conventional hardware setup and a custom training approach to extend the applicability of vision-based pipelines. A laser pointer is introduced in the setup as an effective mechanism for object disambiguation. Accordingly, a suitable retraining of the deep convolutional neural networks supporting recognition of graspable parts is implemented. A mixed reality strategy allows the reuse of available high-quality segmentation masks to generate realistic images procedurally, extending the range of scenarios supported by the final device. Experimental results confirm that the proposed pipeline improves target disambiguation accuracy by up to 9% compared with existing solutions, while preserving the computational cost.