Actions, not gestures: contextualising embodied controller interactions in immersive virtual reality

Описание: Actions, not gestures: contextualising embodied controller interactions in immersive virtual reality
Jack Ratcliffe, Nick Ballou, and Laurissa Tokarchuk

VRST 2021
Session: Paper 7 Input Methods

Abstract
Modern immersive virtual reality (IVR) often uses embodied controllers for interacting with virtual objects. It is not clear, however, how we should conceptualise these interactions - as gestures or actions. These interactions could be categorised as either gestures, as there is no interaction with a physical object, or as actions, as the experience is convincing enough to cognitively substitute for a physical world action. This distinction is important, as in the physical world, literature has shown that sensorimotor-based learning produces distinct cognitive outcomes depending on whether an experience uses an action (stronger memorisation) or a gesture (more generalizable learning). This study attempts to understand whether sensorimotor-embodied controller interactions in IVR can cognitively be considered as actions or gestures. It does this by comparing verb-learning outcomes between two conditions: (1) where participants move the controllers without touching virtual objects (gesture condition); and (2) where participants move the controllers and manipulate virtual objects (action condition). We found that (1) users can have cognitively distinct outcomes in IVR based on whether the interactions are actions or gestures, with actions providing stronger memorisation outcomes; and (2) embodied controller actions in IVR behave more similarly to actions in the physical-world than gestures in terms of verb memorization benefits.

DOI:: https://doi.org/10.1145/3489849.3489892
WEB:: https://vrst.acm.org/vrst2021/

Videos for VRST 2021