User Experience in Human-Machine Interaction: Insights from Field Studies in Autonomous Mobility
2026-08-31 • Human-Computer Interaction
Human-Computer Interaction
AI summaryⓘ
The authors conducted real-world tests with autonomous vehicle passengers to understand how people actually feel when riding in AVs, using tools like heartbeat, breathing, and voice analysis. They found that some measurements, like breathing and voice, work well to judge passenger emotions in natural settings. They also created new ways for passengers to give feedback during rides and better surveys to understand attitudes towards AVs. This study helps make self-driving cars more user-friendly by focusing on real experiences instead of just lab tests or driver-only views.
autonomous vehiclesuser acceptanceaffective computingmulti-modal sensingheartbeat monitoringbreathing analysisvoice signal processinguser experience (UX)real-world field studieshuman-machine interaction
Authors
Helen Schneider, Svetlana Pavlitska, J. Marius Zöllner
Abstract
Autonomous vehicles (AVs) promise safer, cleaner, and more inclusive mobility, yet large-scale adoption is hindered by user acceptance rather than by technical challenges. Prior studies on acceptance and user experience largely rely on surveys, simulators or Wizard-of-Oz setups, often over-representing technologically enthusiastic participants and focusing on drivers instead of passengers. We address this gap with real-world field studies with AVs in real traffic, totaling 144 participants. Using multi-modal sensing, we evaluated EGG, heartbeat, breathing, camera and voice signals for affect inference in combination with vehicle data. Our results show that breathing, camera and voice measurements are reliable and pratical in naturalistic passenger contexts. We further contribute a validated study protocol, a self-assessment app for real-time assessment during human-machine interaction, and a tailored questionnaire to capture participant attitudes towards AVs. By grounding UX evaluation in real-world contexts, this work lays a foundation for user-centered design of autonomous mobility systems and robotics in general. Our work bridges the gap between affective computing and technical implementation of autonomous vehicles.