Wrist sensors enable fast finger touch input for virtual reality
TapID: Rapid touch interaction in virtual reality using wearable sensing
Human-Computer Interaction
Summary
Virtual reality (VR) typically uses cameras to track hand and controller movements, but this can be tiring for long use. The paper presents TapID, a new way to detect when fingers tap surfaces using sensors worn on the wrist. These sensors identify which finger tapped and when, improving input speed and comfort in VR. The authors tested TapID and found it works accurately for finger detection in typical VR table scenarios.
What this means in practice
- •For vr interface designers: Create VR interfaces that respond to rapid finger taps on surfaces using wrist sensors to complement existing hand tracking.
- •For virtual instrument developers: Build virtual musical instruments like pianos that detect individual finger taps via wrist sensors for faster and more natural play in VR.
Authors
Manuel Meier, Paul Streli, Andreas Fender, Christian Holz
Abstract
Current Virtual Reality systems typically use cameras to capture user input from controllers or free-hand mid-air interaction. In this paper, we argue that this is a key impediment to productivity scenarios in VR, which require continued interaction over prolonged periods of time---a requirement that controller or free-hand input in mid-air does not satisfy. To address this challenge, we bring rapid touch interaction on surfaces to Virtual Reality---the input modality that users have grown used to on phones and tablets for continued use. We present TapID, a wrist-based inertial sensing system that complements headset-tracked hand poses to trigger input in VR. TapID embeds a pair of inertial sensors in a flexible strap, one at either side of the wrist; from the combination of registered signals, TapID reliably detects surface touch events and, more importantly, identifies the finger used for touch. We evaluated TapID in a series of userstudies on event-detection accuracy (F1 = 0.997) and hand-agnostic finger-identification accuracy (within-user: F1 = 0.93; across users: F1 = 0.91 after 10 refinement taps and F1 = 0.87 without refinement) in a seated table scenario. We conclude with a series of applications that complement hand tracking with touch input and that are uniquely enabled by TapID, including UI control, rapid keyboard typing and piano playing, as well as surface gestures.