Gaze control speeds moon rock handling with mixed manual precision

Gaze-Assisted Supervisory Control for Lunar Robotic Arm Teleoperation Under Concurrent Monitoring

Robotics

Summary

Using a lunar rover's robotic arm to pick up and place rocks normally requires careful joystick control and watching many things at once, which is slow and tiring. The authors designed a system where looking at a rock and then confirming by hand lets the robot automatically grab and place it, speeding up the work. In tests with people, this gaze-assisted method cut the time per rock from about 48 seconds to 18 seconds, especially speeding up the grabbing part. However, when the task needed very precise placement, manual joystick control still worked better. This suggests combining gaze-based commands for rough moves with manual control for fine details works best.

What this means in practice

  • For space mission operators: Enable faster lunar sample collection by blending gaze-based commands with manual precision during arm teleoperation.
  • For manufacturing robot supervisors: Reduce operator workload by integrating gaze-assisted supervisory control for robotic arms that handle repetitive pick-and-place tasks while monitoring system status.

Authors

Jiayu He, Yifan Li, Oliver R. Runswick, Ernest N. Kamavuako, Yunjie Jia, Peter D. Hodkinson, Yang Gao

Abstract

Rover-mounted robotic arms can extend human reach during lunar sample handling, but continuous joystick teleoperation requires repeated visual-motor alignment while operators may also need to monitor system states. This paper presents a gaze-assisted supervisory control framework in which gaze nominates a rock or placement target, explicit confirmation authorizes the action, and the robot performs bounded autonomous grasp-and-place execution. The framework was evaluated against manual joystick teleoperation in a mission-analog virtual reality study with 16 participants. The study used a 2 x 2 x 3 within-subject design varying control mode, placement difficulty, and concurrent monitoring load. Mean recorded task completion time decreased from 47.65 to 18.43 s per rock, with 72.5% of this difference arising during grasping. Under the tested auditory monitoring conditions, gaze-assisted supervisory control showed less average proportional slowing, with no detected control-mode difference in monitoring-response accuracy or speed. The rate of gaze shifts between core areas of interest decreased from 49.98 to 27.73 shifts/min, while the raw NASA Task Load Index score decreased from 32.54 to 18.26. However, the benefit was task dependent. In precision placement, manual control yielded lower mean placement error and a higher placement success rate. These findings support a mixed-initiative allocation in which gaze enables rapid task-level intention specification and autonomous execution, while precision-critical stages retain manual refinement.