Up to the Finger Tip: The Effect of Avatars on Mid-Air Pointing Accuracy in Virtual Reality

Avatars in virtual reality (VR) increase the immersion and provide an interface between the user's physical body and the virtual world. Thus, avatars enable referential gestures, which are essential for targeting, selection, locomotion, and collaboration in VR. However, players of immersive games can have another virtual appearance deviating from human-likeness and previous work suggests that avatars can have an effect on the accuracy of referential gestures in VR. One of the most important referential gestures is mid-air pointing. It has been shown that mid-air pointing is affected by systematic errors, which can be compensated using different methods. Thus, it is unknown if the avatar must be considered in corrections of the systematic error. In this paper, we investigate the effect of the avatar on pointing accuracy. We show that the systematic error in pointing is significantly affected by the virtual appearance but does not correlate with the degree to which the appearance deviates from the perceived human-likeness. Moreover, we confirm that people only rely on their fingertip and not on their forearm or index finger orientation. We present compensation models and contribute with design implications to increase the accuracy of pointing in VR.

[1]  Valentin Schwind,et al.  Touch with foreign hands: the effect of virtual hand appearance on visual-haptic integration , 2018, SAP.

[2]  Niels Henze,et al.  "These are not my hands!": Effect of Gender on the Perception of Avatar Hands in Virtual Reality , 2017, CHI.

[3]  Ferran Argelaguet,et al.  Overcoming eye-hand visibility mismatch in 3D pointing selection , 2008, VRST '08.

[4]  Mel Slater,et al.  An Investigation of Presence Response across Variations in Visual Realism , 2004 .

[5]  Karl F. MacDorman,et al.  The Uncanny Valley [From the Field] , 2012, IEEE Robotics Autom. Mag..

[6]  L. Camaioni,et al.  The Role of Declarative Pointing in Developing a Theory of Mind , 2004 .

[7]  Frederick P. Brooks,et al.  Moving objects in space: exploiting proprioception in virtual-environment interaction , 1997, SIGGRAPH.

[8]  M. Sheelagh T. Carpendale,et al.  A comparison of ray pointing techniques for very large displays , 2010, Graphics Interface.

[9]  Yael Edan,et al.  Vision-based hand-gesture applications , 2011, Commun. ACM.

[10]  内海 孝信,et al.  私の留学体験記 The University of North Carolina at Chapel Hill , 2019 .

[11]  M. Graziano.  Where is my arm? The relative role of vision and proprioception in the neuronal representation of limb position. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[12]  Daniel Vogel,et al.  Distant freehand pointing and clicking on very large, high resolution displays , 2005, UIST.

[13]  Enrico Rukzio,et al.  Towards accurate cursorless pointing: the effects of ocular dominance and handedness , 2018, Personal and Ubiquitous Computing.

[14]  Philip R. Cohen,et al.  Multimodal speech-gesture interface for handfree painting on a virtual paper using partial recurrent neural networks as gesture recognizer , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[15]  W. Miles.  Ocular dominance in human adults. , 1930 .

[16]  Niels Henze,et al.  Physical Keyboards in Virtual Reality: Analysis of Typing Performance and Effects of Avatar Hands , 2018, CHI.

[17]  M S Graziano,et al.  Coding the location of the arm by sight. , 2000, Science.

[18]  A. Kendon.  Gesture: Visible Action as Utterance , 2004 .

[19]  Niels Henze,et al.  Modeling Distant Pointing for Compensating Systematic Displacements , 2015, CHI.

[20]  Ferran Argelaguet,et al.  The role of interaction in virtual embodiment: Effects of the virtual hand representation , 2016, 2016 IEEE Virtual Reality (VR).

[21]  Sandra G. Hart,et al.  Nasa-Task Load Index (NASA-TLX); 20 Years Later , 2006 .

[22]  R. Passingham,et al.  That's My Hand! Activity in Premotor Cortex Reflects Feeling of Ownership of a Limb , 2004, Science.

[23]  Marc Erich Latoschik,et al.  Conversational Pointing Gestures for Virtual Reality Interaction: Implications from an Empirical Study , 2008, 2008 IEEE Virtual Reality Conference.

[24]  S Morrison,et al.  Changes in the dynamics of tremor during goal-directed pointing. , 2001, Human movement science.

[25]  Niels Henze,et al.  Is there an uncanny valley of virtual animals? A quantitative and qualitative investigation , 2018, Int. J. Hum. Comput. Stud..

[26]  Mel Slater,et al.  Taking steps: the influence of a walking technique on presence in virtual reality , 1995, TCHI.

[27]  Mark R. Mine,et al.  Virtual Environment Interaction Techniques , 1995 .

[28]  Maria V. Sanchez-Vives,et al.  Inducing Illusory Ownership of a Virtual Body , 2009, Front. Neurosci..

[29]  Tiziana Aureli,et al.  The Pointing-Vocal Coupling Progression in the First Half of the Second Year of Life , 2017 .

[30]  Richard A. Bolt,et al.  “Put-that-there”: Voice and gesture at the graphics interface , 1980, SIGGRAPH '80.

[31]  Rainer Stiefelhagen,et al.  Implementation and evaluation of a constraint-based multimodal fusion system for speech and 3D pointing gestures , 2004, ICMI '04.

[32]  Dan R. Olsen,et al.  Laser pointer interaction , 2001, CHI.

[33]  John B. Haviland,et al.  How to Point in Zinacantán , 2003 .

[34]  Heloir,et al.  The Uncanny Valley , 2019, The Animation Studies Reader.

[35]  Niels Henze,et al.  "Where's Pinky?": The Effects of a Reduced Number of Fingers in Virtual Reality , 2017, CHI PLAY.

[36]  Mel Slater,et al.  Drumming in Immersive Virtual Reality: The Body Shapes the Way We Play , 2013, IEEE Transactions on Visualization and Computer Graphics.

[37]  Rainer Stiefelhagen,et al.  Pointing gesture recognition based on 3D-tracking of face, hands and head orientation , 2003, ICMI '03.

[38]  Thies Pfeiffer,et al.  Measuring and Reconstructing Pointing in Visual Contexts , 2006 .

[39]  Poika Isokoski,et al.  Accuracy of interpreting pointing gestures in egocentric view , 2016, UbiComp.

[40]  Karl F. MacDorman,et al.  Refinements to Indices for Perceived Humanness, Attractiveness, and Eeriness , 2016 .

[41]  Jeffrey Nichols,et al.  Interacting at a distance: measuring the performance of laser pointers and other devices , 2002, CHI.

[42]  Sophie Jörg,et al.  Need a hand?: how appearance affects the virtual hand illusion , 2016, SAP.

[43]  John M. Foley,et al.  Visually directed pointing as a function of target distance, direction, and available cues , 1972 .

[44]  Marko Jurmu,et al.  Scroll, tilt or move it: using mobile phones to continuously control pointers on large public displays , 2009, OZCHI.

[45]  Niels Henze,et al.  The Effect of Offset Correction and Cursor on Mid-Air Pointing in Real and Virtual Environments , 2018, CHI.

[46]  Marc Erich Latoschik,et al.  Avatar anthropomorphism and illusion of body ownership in VR , 2015, 2015 IEEE Virtual Reality (VR).