Multimodal system tracks workload in human robot physical interactions
Online Multimodal Workload Assessment in Contact-Rich Physical Human-Robot Interaction
Robotics
Summary
Physical work between humans and robots often changes how hard someone is working. The authors created a system that looks at forces, movement, and skin responses to guess how demanding the task feels in real time. They tested this on people doing different tasks in different conditions and showed it matches well with eye pupil size, a known stress indicator. Their method is clear and understandable compared to complex AI models, making it easier to trust and use.
What this means in practice
- •For robot system developers: Incorporate transparent workload indicators to improve human-robot collaboration safety and efficiency during physical interaction tasks.
- •For ergonomics specialists: Use continuous workload assessment that integrates physical and physiological signals to better design tasks and environments involving human-robot contact.
Authors
Yanyi Chen, Fan Yang, Min Deng
Abstract
Contact-rich physical human--robot interaction (pHRI) imposes time-varying demands associated with physical interaction, motor regulation, and physiological response, motivating continuous assessment of interaction workload. This paper presents an online multimodal assessment framework that integrates interaction wrench, planar tool-center-point (TCP) kinematics, and skin conductance level (SCL) into four interpretable workload-related factors. Their relative contributions are adjusted using path curvature to reflect changes in motion demand and task progression to account for gradual physiological variation over time. The framework was evaluated with 24 participants across 18 controlled combinations of temperature, acoustic noise, and illuminance under two admittance-control modes. Strict leave-one-subject-out (LOSO) evaluation used standardized pupil diameter ($\mathrm{PD}_z$) as an independent physiological reference and included comparisons with static variants and representative state-of-the-art learning-based baselines. The proposed framework achieves a cohort-mean $30\,\mathrm{s}$ block-wise Spearman correlation of $ρ_{30}=0.308$ with the physiological reference, with positive subject-level correspondence in 23 of 24 participants. Its overall performance is comparable to the state-of-the-art learning-based baseline. At the same time, our framework keeps the assessment process transparent through explicit workload-related factors and defined weighting rules, while outperforming the corresponding fixed-weight formulation. The framework also maintains consistent performance across the two tested admittance-control modes. These results support a transparent and interpretable approach to continuous interaction workload assessment in contact-rich pHRI.