Screen lets users physically touch virtual agents with variable resistance

Touching Projected Agents through the DAT Screen -System Design and Questions of Interaction Boundaries-

Human-Computer InteractionHardware ArchitectureRobotics

Summary

Touching a virtual character on a screen is more complex than just sensing a finger touch. The researchers built a special screen that can bend and push back against a user's finger, making it feel more realistic. Their screen, called the Desktop Amid Touch (DAT), uses a stretchy mesh that you can push, and motors adjust how hard it is to press. The team explores how such touchable agents might react to being touched and whether they should have 'feelings' or 'boundaries' about touch. They describe their system but don't yet know how people interpret a virtual agent’s acceptance or refusal of touch.

What this means in practice

  • For user interface designers: Develop touchscreens that provide physical resistance to enhance virtual agent interactions in desktop environments.
  • For virtual reality developers: Integrate motorized tension control with touch sensing to create more immersive virtual agent experiences that simulate physical boundaries.

A position paper. It proposes an approach and reports no results.

Authors

Takaya Tamamoto, Tomoko Yonezawa

Abstract

Touching a virtual agent involves more than detecting a finger on a display: the contact surface can deform and resist the user's push, and the agent can respond to that action. We describe the Desktop Amid Touch (DAT) screen, which combines spatial projection onto a semi-transparent, deformable mesh with touch sensing and motorized control of resistance to indentation. An initial prototype detected a single touch and estimated its position and depth from tension changes around a flexible projection screen. A subsequent prototype added motorized tension control to vary the screen's resistance to indentation. We describe the tension-controlled DAT implementation and its demonstrations, distinguishing what has been observed from what remains an open design question. These capabilities raise a design question: should touchable virtual agents respond as though they had preferences or boundaries, and under what conditions should such behavior be implemented on systems like DAT. The present paper describes a system and frames this question; it does not establish how users interpret an agent's refusal or consent.
View PDFOpen arXiv