Open smart ring enables speech and gesture control of robots
OpenZilo: An Open Smart-Ring Platform for Human-Centric Sensing and Interaction with Robots
RoboticsHuman-Computer Interaction
Summary
Controlling robots quickly and naturally is hard, so the authors created an open smart ring that can listen to speech, sense hand gestures, and monitor body signals. They tested its accuracy and speed with 10 people performing robot tasks using speech or gestures. Both ways worked well, with speech slightly more accurate, but gestures were a bit faster in some steps. The study helps understand how wearable devices can connect people and robots effectively through different types of input.
What this means in practice
- •For wearable device developers: Build smart rings that combine speech, gesture, and physiological sensing to control robots across task stages.$Commercial implications: Enables commercial smart-ring products offering multimodal robot control for industrial, medical, or assistive applications.
- •For robotics software engineers: Design robot interaction policies separating event acquisition and execution to improve timing and recognition accuracy with wearable inputs.
Authors
Qianggang Ding, Xingyao Wang, Rui Feng, Chi Qin, Weijian Lin, Lizhi Ding, Jiankai Tang, Yuntao Wang, Xiangke Liu, Bang Liu
Abstract
Wearable robot interfaces must connect user input to timely physical action. We present the proposed platform, an open smart-ring platform for studying this connection across task stages. The ring integrates a microphone, an inertial measurement unit, photoplethysmography, and skin-temperature sensing. Its software architecture separates acquisition, event interpretation, interaction policy, and robot execution. We characterize seven operating modes and evaluate speech and gesture control with 10 participants. Across five commands and 250 trials per modality, speech and gestures achieved task success rates of 90.4% and 88.8%, respectively. Recognition accuracy exceeded task success for both modalities. Gaussian hidden Markov models achieved higher accuracy than a support vector machine across three vocabulary sizes, with longer scoring times. Following a shared spoken task start, gesture-based adjustments had shorter median response times than speech. Whole-task success was 70.0% with gestures and 64.0% with speech. We further examine physiological-event-triggered robot interaction using simulated event notifications. Mean notification-receipt-to-arrival times were 11.71 s for the robot and 9.20 s for an on-site person. These descriptive findings support evaluating wearable robot interfaces through both recognition and physical outcomes, with timing measures appropriate to each task stage.