Pose cues improve robot anticipation of human interaction across sites
Fixed-Reference Pose Residuals for Measuring Cross-Dataset Cue Transfer in Human-Robot Interaction Anticipation
Robotics
Summary
Robots in public spaces need to guess who might soon touch them to respond in time. The authors studied how well cues from one robot’s data work on another robot’s location. They created a method that breaks predictions into parts based on body shape and pose. They found that pose information helped when going from one dataset to another in one direction but not the other. Overall, the method helps measure cue transferability but does not always improve prediction accuracy.
What this means in practice
- •For robot developers: Understand which body pose cues are transferable for anticipating human touch across different robot deployments.
- •For security system designers: Use geometry and pose cues to partially anticipate human approach for improved interaction responsiveness in shared public spaces.
Authors
Bowen Yang, Xinliang Xiao, Wenjing Zhang, Li Yang, Wei Zhou
Abstract
Social and service robots in public spaces need to anticipate which nearby person is about to approach and touch them, so that a response can be prepared before contact. It is largely unknown which cues support this anticipation when a model trained with one robot is used on another robot at a different site. We study this question with a fixed-reference pose residual (FRPR) model: a geometry predictor built from the person's bounding box and mask is trained and frozen, and a temporal network then learns from body pose an additive correction to its logit, so that every prediction splits exactly into a geometry term and a pose term. Between two public egocentric datasets recorded by different robots, HUI360 and SSUP-A, with every choice made on source data, the pose correction raised average precision (AP) from 0.277 to 0.321 from SSUP-A to HUI360 and gave no measurable gain in the opposite direction; the same asymmetry held over a stronger, source-selected geometry reference. Freezing gave no AP advantage over joint training, and simple geometric baselines and tree ensembles remained competitive or better, so the construction serves measurement rather than prediction accuracy. A head-orientation residual added small gains in both directions. Post hoc, whether a person faces the camera kept its discriminative direction across datasets, whereas head pitch reversed. With thresholds chosen on source data, the neural models that use geometry detected at most 17% of target interactions. Code and processed data are available at https://github.com/WeiZhou96/FRPR-interaction-anticipation.