Remote expert controls avatar gaze for better mixed reality teamwork
Remote Expert Evaluation of Tabletop Avatar Gaze Control in Asymmetric Multi-Scale MR--VR Collaboration
Human-Computer Interaction
Summary
Detecting problems collaboratively in remote settings can be tricky when one expert sees things differently than the person on site. This paper describes how the expert can use a special setup with virtual and mixed reality to see a small model of the workspace and control their avatar’s gaze to work better with the on-site operator. The authors tested three ways for experts to control where their avatar looks and found that using full virtual reality with head movements gave the best results. This helps remote experts feel more involved and improves teamwork for tasks like fault detection.
What this means in practice
- •For industrial maintenance teams: Enable remote experts to more accurately guide on-site operators through fault detection using immersive VR avatar gaze control.
- •For remote facility management teams: Use tabletop mixed reality setups to allow remote supervisors to monitor and interact with multiple machines at different scales effectively.
Authors
Carlos Mosquera, Neven Elsayed, Ernst Kruijff, Joseph Newman, Valeriya Lehrbaum, Eduardo Veas
Abstract
This paper proposes an integrated setup for a multi-scale asymmetric collaboration scenario in the context of a remote-expert use case, focusing on the expert side. In this system, the remote expert (with an exocentric view in Mixed Reality (MR)) uses a Tangible User Interface (TUI) to precisely position and rotate their avatar within a scaled tabletop World-in-Miniature (WIM). Inside the WIM, the expert's avatar and the operator share the same scale and egocentric view, while the expert retains a global tabletop overview of the plant to support collaboration. Interviews with industry experts during the participatory design phase revealed the need for an egocentric view inside the plant on the remote-expert side to enable full collaboration. We designed and studied three interaction-visualisation combinations to integrate and control the expert avatar's gaze: VR (fully immersive with direct head control), Head (floating screen with direct head control), and Touch (on-avatar screen with touchpad control). We evaluated these methods with 29 participants across two fault-detection tasks: Same Machine, where both collaborators inspect the same machine, and Twin Machine, where they inspect identical machines in separate locations. The VR condition, which combines immersion, direct head control, and a full-scale view, yielded the strongest overall performance and user experience, while the tabletop avatar setup was well received for collaboration. Our findings inform the design of gaze-control techniques for asymmetric MR-VR collaboration and reveal challenges and opportunities for future work.