Optimization of sim-to-real transfer in the humanoid robot NICO
2026-07-20 • Robotics
Robotics
AI summaryⓘ
The authors improved how a humanoid robot named NICO picks up objects on a table. They used a mix of camera-based object and hand detection, stereo vision for locating objects, and special corrections to make the robot’s grasp more accurate. Their method doesn't need expensive or complex equipment like RGB-D cameras or motion tracking. They also added a system that visually aligns the robot's hand with the object before trying to pick it up. Their tests showed that the new calibration model worked best in certain areas, while the visual alignment worked best overall.
robotic graspingsim-to-real gapYOLO object detectionstereo visionfisheye camerasinverse kinematicsvisual feedbackcalibration modelhumanoid robothand-eye coordination
Authors
Juraj Gavura, Igor Farkaš
Abstract
Robotic grasping requires accurate coordination between visual perception, object localization, inverse kinematics, and hand control. However, when movements planned in simulation are executed on a physical robot, the sim-to-real gap can cause small positioning errors that prevent successful grasping. In our previous work, we introduced a low-cost haptic calibration method that improved 2D reaching accuracy of the humanoid robot NICO. In this paper, we extend this approach from reaching to tabletop object grasping by adding YOLO-based object and hand detection, stereo vision-based localization using the robot's built-in low-resolution fisheye cameras, and task-specific corrections for grasp execution. Together, these components form a novel calibration-based grasping pipeline that does not require RGB-D cameras, motion capture, or external tracking systems. We also implemented a visual feedback model that aligns the robot hand with the detected object before grasping. Our results show that the fully nonlinear calibration model achieved the best performance inside the calibrated area, while the visual feedback model achieved the highest overall grasping success across the full tabletop workspace.