Bedside robot asks questions before alerting for distress cues
Ask Before It Tells: Benchmark-to-Robot Body-Cue Transfer for a Question-First Bedside Robot
RoboticsHuman-Computer Interaction
Summary
Recognizing signs of distress is important for assistive robots, but what works well in lab tests may not work well from a robot’s camera view. The authors built a bedside robot called Nuni that asks questions when it sees a possible distress signal instead of immediately raising an alarm. They showed that a hybrid method focusing on body pose recognized distress better from the robot’s viewpoint than appearance-based methods. By asking first, the robot can avoid false alarms and only alert when needed. This approach helps manage uncertainty in robot perception during interactions.
What this means in practice
- •For robotics engineers: Implement body-cue recognition with question-first interaction policies to reduce false alarms in assistive bedside robots.
- •For healthcare technology developers: Design assistive robots that improve patient monitoring by confirming detected distress signals through interaction before alerting caregivers.
Authors
Dongsik Yoon
Abstract
Body-cue recognition can support assistive robots, but benchmark accuracy does not guarantee reliable behavior under a robot-camera viewpoint. We present Nuni, a bedside robot prototype that treats a detected distress cue as a reason to ask rather than a reason to alert. We compare two X3D-UGT RGB appearance classifiers, which reach 97.7% and 94.8% six-way accuracy on NTU RGB+D, with a pose-centric hybrid pipeline on 28 single-actor scripted clips recorded from the robot camera. The hybrid path achieved 0.71 six-way macro recall, versus 0.25 and 0.29 for the fine-tuned and from-scratch RGB variants. More importantly for interaction, it produced a question-triggering distress cue in 12/16 distress clips and would have prompted unnecessarily in 2/8 normal clips; the RGB variants yielded a question-triggering cue in only 2/16 and 3/16 distress clips. We separately tested the question-first controller through event injection. All 13 state-transition trials passed: valid responses caused stand-down, two unanswered prompts produced one alert, and three boundary conditions were handled correctly. These results are a preliminary technical evaluation, not a user study or medical validation, but they show how interaction policy can limit the consequences of uncertain perception.