Evaluating the Effects of Hand-gesture-based Interaction with Virtual Content in a 360° Movie

Head-mounted displays are becoming increasingly popular as home entertainment devices for viewing 360◦ movies. This paper explores the effects of adding gesture interaction with virtual content and two different hand-visualisation modes for 360◦ movie watching experience. The system in the study comprises of a Leap Motion sensor to track the user’s hand and finger motions, in combination with a SoftKinetic RGB-D camera to capture the texture of the hands and arms. A 360◦ panoramic movie with embedded virtual objects was used as content. Four conditions, displaying either a point-cloud of the real hand or a rigged computer-generated hand, with and without interaction, were evaluated. Presence, agency, embodiment, and ownership, as well as the overall participant preference were measured. Results showed that participants had a strong preference for the conditions with interactive virtual content, and they felt stronger embodiment and ownership. The comparison of the two hand visualisations showed that the display of the real hand elicited stronger ownership. There was no overall difference for presence between the four conditions. These findings suggest that adding interaction with virtual content could be beneficial to the overall user experience, and that interaction should be performed using the real hand visualisation instead of the virtual hand if higher ownership is desired.

[1]  Ferran Argelaguet,et al.  The role of interaction in virtual embodiment: Effects of the virtual hand representation , 2016, 2016 IEEE Virtual Reality (VR).

[2]  Mel Slater,et al.  Using Presence Questionnaires in Reality , 2000, Presence: Teleoperators & Virtual Environments.

[3]  Holger Regenbrecht,et al.  The Experience of Presence: Factor Analytic Insights , 2001, Presence: Teleoperators & Virtual Environments.

[4]  Sophie Jörg,et al.  Need a hand?: how appearance affects the virtual hand illusion , 2016, SAP.

[5]  Klaus H. Hinrichs,et al.  Enhancing Presence in Head-Mounted Display Environments by Visual Body Feedback Using Head-Mounted Cameras , 2009, 2009 International Conference on CyberWorlds.

[6]  Fumihisa Shibata,et al.  Design and Prototype Implementation of MR Pre-Visualization Workflow , 2006 .

[7]  Hirokazu Kato,et al.  Interactive Theatre Experience in Embodied + Wearable Mixed Reality Space , 2002, ISMAR.

[8]  Mary C. Whitton,et al.  Effects of handling real objects and avatar fidelity on cognitive task performance in virtual environments , 2003, IEEE Virtual Reality, 2003. Proceedings..

[9]  M. Krijn,et al.  Treatment of acrophobia in virtual reality: the role of immersion and presence. , 2004, Behaviour research and therapy.

[10]  Jihad El-Sana,et al.  Free-hand interaction for handheld augmented reality using an RGB-depth camera , 2013, SA '13.

[11]  Roderick Murray-Smith,et al.  A Dose of Reality: Overcoming Usability Challenges in VR Head-Mounted Displays , 2015, CHI Extended Abstracts.

[12]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Fumihisa Shibata,et al.  On-site real-time 3D match move for MR-based previsualization with relighting , 2010, SIGGRAPH '10.

[14]  J. J. Higgins,et al.  The aligned rank transform for nonparametric factorial analyses using only anova procedures , 2011, CHI.

[15]  Mary C. Whitton,et al.  Walking > walking-in-place > flying, in virtual environments , 1999, SIGGRAPH.