Impact of Information Placement and User Representations in VR on Performance and Embodiment

Human sensory processing is sensitive to the proximity of stimuli to the body. It is therefore plausible that these perceptual mechanisms also modulate the detectability of content in VR, depending on its location. We evaluate this in a user study and further explore the impact of the user's representation during interaction. We also analyze how embodiment and motor performance are influenced by these factors. In a dual-task paradigm, participants executed a motor task, either through virtual hands, virtual controllers, or a keyboard. Simultaneously, they detected visual stimuli appearing in different locations. We found that, while actively performing a motor task in the virtual environment, performance in detecting additional visual stimuli is higher when presented near the user's body. This effect is independent of how the user is represented and only occurs when the user is also engaged in a secondary task. We further found improved motor performance and increased embodiment when interacting through virtual tools and hands in VR, compared to interacting with a keyboard. This article contributes to better understanding the detectability of visual content in VR, depending on its location in the virtual environment, as well as the impact of different user representations on information processing, embodiment, and motor performance.

[1]  Andrea Serino,et al.  Suppression of premotor cortex disrupts motor coding of peripersonal space , 2012, NeuroImage.

[2]  Takuji Narumi,et al.  Virtual Hand Realism Affects Object Size Perception in Body-Based Scaling , 2019, 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[3]  Miguel A. Nacenta,et al.  Quantitative measurement of virtual vs. physical object embodiment through kinesthetic figural after effects , 2014, CHI.

[4]  A. Maravita,et al.  Tools for the body (schema) , 2004, Trends in Cognitive Sciences.

[5]  Anatole Lécuyer,et al.  Virtual Objects Look Farther on the Sides: The Anisotropy of Distance Perception in Virtual Reality , 2019, 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[6]  Pamela J. Wisniewski,et al.  Over My Hand: Using a Personalized Hand in VR to Improve Object Size Estimation, Body Ownership, and Presence , 2018, SUI.

[7]  Alessandro Farnè,et al.  Tool-use: An open window into body representation and its plasticity , 2016, Cognitive neuropsychology.

[8]  Yu Sun,et al.  The Effect of Hand Size and Interaction Modality on the Virtual Hand Illusion , 2019, 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[9]  R. Baayen,et al.  Analyzing Reaction Times , 2010 .

[10]  Mar Gonzalez-Franco,et al.  Avatar Embodiment Enhances Haptic Confidence on the Out-of-Body Touch Illusion , 2019, IEEE Transactions on Haptics.

[11]  Cristy Ho,et al.  Using Peripersonal Warning Signals to Orient a Driver’s Gaze , 2009, Hum. Factors.

[12]  Christopher C. Pagano,et al.  Investigating the Effects of Anthropomorphic Fidelity of Self-Avatars on Near Field Depth Perception in Immersive Virtual Environments , 2018, 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[13]  Sian L. Beilock,et al.  When paying attention becomes counterproductive: impact of divided versus skill-focused attention on novice and experienced performance of sensorimotor skills. , 2002, Journal of experimental psychology. Applied.

[14]  Miguel A. Nacenta,et al.  Quantitative Measurement of Tool Embodiment for Virtual Reality Input Alternatives , 2019, CHI.

[15]  G. Pellegrino,et al.  Peripersonal space in the brain , 2015, Neuropsychologia.

[16]  Jörg Müller,et al.  User Representations in Human-Computer Interaction , 2020, Hum. Comput. Interact..

[17]  Emiliano Macaluso,et al.  Orienting of visuo‐spatial attention in complex 3D space: Search and detection , 2015, Human brain mapping.

[18]  Gordon Wetzstein,et al.  Optimizing virtual reality for all users through gaze-contingent and adaptive focus displays , 2017, Proceedings of the National Academy of Sciences.

[19]  Maria V. Sanchez-Vives,et al.  Inducing Illusory Ownership of a Virtual Body , 2009, Front. Neurosci..

[20]  Anatole Lécuyer,et al.  Design and Application of Real-Time Visual Attention Model for the Exploration of 3D Virtual Environments , 2012, IEEE Transactions on Visualization and Computer Graphics.

[21]  Joan López-Moliner,et al.  The sense of body ownership relaxes temporal constraints for multisensory integration , 2016, Scientific Reports.

[22]  Sandra G. Hart,et al.  Nasa-Task Load Index (NASA-TLX); 20 Years Later , 2006 .

[23]  Jonathan D. Cohen,et al.  Rubber hands ‘feel’ touch that eyes see , 1998, Nature.

[24]  Liana E Brown,et al.  Agency and Control for the Integration of a Virtual Tool into the Peripersonal Space , 2011, Perception.

[25]  C Lopez Maïté,et al.  Ecological assessment of divided attention: What about the current tools and the relevancy of virtual reality. , 2016, Revue neurologique.

[26]  Heinrich H. Bülthoff,et al.  The influence of avatar (self and character) animations on distance estimation, object interaction and locomotion in immersive virtual environments , 2011, APGV '11.

[27]  Tabitha C. Peck,et al.  Avatar Embodiment. Towards a Standardized Questionnaire , 2018, Front. Robot. AI.

[28]  Mel Slater,et al.  The Sense of Embodiment in Virtual Reality , 2012, PRESENCE: Teleoperators and Virtual Environments.

[29]  Bobby Bodenheimer,et al.  Determining Peripersonal Space Boundaries and Their Plasticity in Relation to Object and Agent Characteristics in an Immersive Virtual Environment , 2020, 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[30]  Sally Andrews,et al.  To transform or not to transform: using generalized linear mixed models to analyse reaction time data , 2015, Front. Psychol..

[31]  Olaf Blanke,et al.  From multisensory integration in peripersonal space to bodily self‐consciousness: from statistical regularities to statistical inference , 2018, Annals of the New York Academy of Sciences.

[32]  Kasper Hornbæk,et al.  Tool Extension in Human-Computer Interaction , 2019, CHI.

[33]  Andrea Serino,et al.  Everyday use of the computer mouse extends peripersonal space representation , 2010, Neuropsychologia.

[34]  M. Slater,et al.  Measuring the Effects through Time of the Influence of Visuomotor and Visuotactile Synchronous Stimulation on a Virtual Body Ownership Illusion , 2014, Perception.

[35]  Ferran Argelaguet,et al.  The role of interaction in virtual embodiment: Effects of the virtual hand representation , 2016, 2016 IEEE Virtual Reality (VR).

[36]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[37]  Heinrich H. Bülthoff,et al.  The Effect of Viewing a Self-Avatar on Distance Judgments in an HMD-Based Virtual Environment , 2010, PRESENCE: Teleoperators and Virtual Environments.

[38]  Andrea Serino,et al.  Action-dependent plasticity in peripersonal space representations , 2008, Cognitive neuropsychology.

[39]  Gregory D. Abowd,et al.  NotifiVR: Exploring Interruptions and Notifications in Virtual Reality , 2018, IEEE Transactions on Visualization and Computer Graphics.

[40]  M. Slater,et al.  Illusory ownership of a virtual child body causes overestimation of object sizes and implicit attitude changes , 2013, Proceedings of the National Academy of Sciences.

[41]  Jörg Müller,et al.  Control Theoretic Models of Pointing , 2017, ACM Trans. Comput. Hum. Interact..

[42]  Niels Henze,et al.  Notification in VR: The Effect of Notification Placement, Task and Environment , 2019, CHI PLAY.

[43]  Rui Ni,et al.  Limits of spatial attention in three-dimensional space and dual-task driving performance. , 2011, Accident; analysis and prevention.

[44]  Anthony Steed,et al.  The impact of a self-avatar on cognitive load in immersive virtual reality , 2016, 2016 IEEE Virtual Reality (VR).

[45]  Jeanine Stefanucci,et al.  Egocentric distance perception in the Oculus Rift (DK2) , 2015, SAP.

[46]  Bruno Herbelin,et al.  Peripersonal Space: An Index of Multisensory Body–Environment Interactions in Real, Virtual, and Mixed Realities , 2018, Front. ICT.

[47]  Andrew J. Doxon,et al.  Peri-personal space as a prior in coupling visual and proprioceptive signals , 2018, Scientific Reports.

[48]  Patrick Marmaroli,et al.  Body part-centered and full body-centered peripersonal space representations , 2015, Scientific Reports.

[49]  Valentin Schwind,et al.  Touch with foreign hands: the effect of virtual hand appearance on visual-haptic integration , 2018, SAP.

[50]  Jaka Sodnik,et al.  Detection-Response Task—Uses and Limitations , 2018, Sensors.

[51]  J. Qian,et al.  Working memory for stereoscopic depth is limited and imprecise—evidence from a change detection task , 2019, Psychonomic Bulletin & Review.

[52]  Takuji Narumi,et al.  Do You Feel Like Passing Through Walls?: Effect of Self-Avatar Appearance on Facilitating Realistic Behavior in Virtual Environments , 2020, CHI.

[53]  Andrea Serino,et al.  Peripersonal space (PPS) as a multisensory interface between the individual and the environment, defining the space of the self , 2019, Neuroscience & Biobehavioral Reviews.

[54]  A. Berti,et al.  When Far Becomes Near: Remapping of Space by Tool Use , 2000, Journal of Cognitive Neuroscience.

[55]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[56]  James D. Hollan,et al.  Direct Manipulation Interfaces , 1985, Hum. Comput. Interact..

[57]  William Buxton,et al.  When it gets more difficult, use both hands: exploring bimanual curve manipulation , 2005, Graphics Interface.

[58]  Maria Roussou,et al.  Effects of Virtual Hand Representation on Interaction and Embodiment in HMD-based Virtual Environments Using Controllers , 2020, 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[59]  Niels Henze,et al.  Physical Keyboards in Virtual Reality: Analysis of Typing Performance and Effects of Avatar Hands , 2018, CHI.

[60]  Konrad Paul Kording,et al.  Over my fake body: body ownership illusions for studying the multisensory basis of own-body perception , 2015, Front. Hum. Neurosci..

[61]  Ehud Zohary,et al.  Is That Near My Hand? Multisensory Representation of Peripersonal Space in Human Intraparietal Sulcus , 2007, The Journal of Neuroscience.

[62]  Marc Erich Latoschik,et al.  Any “Body” There? Avatar Visibility Effects in a Virtual Reality Game , 2018, 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).