Framework benchmarks context aware extended reality interface suggestions
A Benchmarking Framework for Context-aware XR Interfaces
Human-Computer InteractionArtificial Intelligence
Summary
Extended Reality (XR) systems can change what tools they show you depending on where you are and what you're doing, but it's hard to measure how well they do this. The authors created ContextXR, a way to break down XR apps into groups of related functions to better test suggestion methods. They also made a dataset called MineXR++ to support this testing and defined tasks to see how well systems suggest the right tools. Their experiments show that ContextXR lets developers consistently compare different methods for recommending XR features based on context.
What this means in practice
- •For xr interface developers: Evaluate and compare ways to suggest functional groups in XR apps depending on user context.
- •For virtual reality content creators: Use the framework to understand how users navigate and find features in immersive environments.
Authors
Hyunsung Cho, Sarah Yewon Yun, Nancy Ruonan Sun, Ben Lafreniere, Mark Parent, Kashyap Todi, Tanya R. Jonker, Hrvoje Benko, Sherry Tongshuang Wu, David Lindlbauer
Abstract
Everyday Extended Reality (XR) systems aim to provide context-aware access to the right functionalities at the right time and place, with minimal manual reconfiguration as users switch context. Yet these interfaces are hard to evaluate: current prototyping and user-study workflows offer no systematic, repeatable way to compare adaptation methods across users and scenarios. We present ContextXR, a novel benchmarking framework for context-aware XR interfaces. ContextXR represents an XR application as a connected graph of functional facets, each a semantically coherent group of related capabilities that together support a shared user intent. On this representation, we build MineXR++, a dataset augmenting prior XR interface data with facet-level annotations, and formulate three canonical tasks of context-aware suggestion: context factor analysis, initial facet suggestion, and next facet suggestion. Our evaluation protocol scores suggestion methods by a simulated interaction metric, the navigation and search cost of reaching the desired functionality. Through experiments benchmarking global popularity, relational retrieval, and LLM-based methods, we demonstrate that ContextXR enables the systematic, reproducible evaluation of context-aware XR interfaces.