Embodiment and Performance in the Supernumerary Hand Illusion in Augmented Reality

In teleoperations, robots are generally used because related tasks are too dangerous, uncomfortable or impossible for humans to perform. When using augmented reality to control robotic limbs in such teleoperations, it is essential to understand how these extra virtual limbs are experienced. In particular, the relationship between the embodiment experience of the user and relevant outcomes such as task performance must be examined. In this article, we study the relationship between experienced embodiment of a supernumerary virtual arm that acts alongside a user’s two real arms, and their task performance in augmented reality. Specifically, we compare how well users can trace a virtual half ring placed just outside of personal space using their virtual arm in a condition where there is expected to be low embodiment (a floating disconnected hand) and a condition where there is expected to be high embodiment (a connected arm and hand). Embodiment is measured quantitatively through skin conductance response and qualitatively through ownership, agency, and self-location questionnaires. Performance is measured in terms of tracing precision. The results show positive correlations between subjective ownership and agency, and agency and performance, but no correlation between subjective or objective ownership and performance. Also, ownership ratings were low overall, while the agency ratings were significantly higher for the disconnected hand condition than the connected arm condition, as was performance. Notably, the presence of the virtual arm evoked incorrect expectations of the movement capabilities of the arm, which may have contributed to an overall preference for the unrealistic disconnected hand over the more realistic connected arm in this particular task. Our results imply that methods to increase performance in various teleoperations can indeed be found in the experience of embodiment: not necessarily directly through ownership, but through ownership mediated by agency.

[1]  Fotis Liarokapis,et al.  Examining the effect of body ownership in immersive virtual and augmented reality environments , 2016, The Visual Computer.

[2]  Hooman Hedayati,et al.  Improving Collocated Robot Teleoperation with Augmented Reality , 2018, 2018 13th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[3]  Kangsoo Kim,et al.  The impact of avatar-owner visual similarity on body ownership in immersive virtual reality , 2017, VRST.

[4]  Jonathan D. Cohen,et al.  Rubber hands ‘feel’ touch that eyes see , 1998, Nature.

[5]  Mel Slater,et al.  Body ownership increases the interference between observed and executed movements , 2019, PloS one.

[6]  Mel Slater,et al.  The Sense of Embodiment in Virtual Reality , 2012, PRESENCE: Teleoperators and Virtual Environments.

[7]  Robert B. Welch,et al.  How Can We Determine if the Sense of Presence Affects Task Performance? , 1999, Presence.

[8]  P. Verschure,et al.  Visuotactile integration modulates motor performance in a perceptual decision-making task , 2017, Scientific Reports.

[9]  S. Debener,et al.  The Senses of Agency and Ownership: A Review , 2018, Front. Psychol..

[10]  Peter J. Werkhoven,et al.  The Supernumerary Hand Illusion in Augmented Reality , 2019, ACM Trans. Appl. Percept..

[11]  Jörg Müller,et al.  Extending the Body for Interaction with Reality , 2017, CHI.

[12]  Remco C. Veltkamp,et al.  Player-Avatar Link: Interdisciplinary Embodiment Perspectives , 2019, Encyclopedia of Computer Graphics and Games.

[13]  Heloir,et al.  The Uncanny Valley , 2019, The Animation Studies Reader.

[14]  P. Haggard,et al.  Sense of agency , 2012, Current Biology.

[15]  J. Bakdash,et al.  Repeated Measures Correlation , 2017, Front. Psychol..

[16]  Karl F. MacDorman,et al.  The Uncanny Valley [From the Field] , 2012, IEEE Robotics Autom. Mag..

[17]  Jeremy N. Bailenson,et al.  Evaluating Control Schemes for the Third Arm of an Avatar , 2016, PRESENCE: Teleoperators and Virtual Environments.

[18]  Sule Serubugo,et al.  Extending the human body in virtual reality: effect of sensory feedback on agency and ownership of virtual wings , 2016, VRIC.

[19]  Yvonne de Kort,et al.  Is This My Hand I See Before Me? The Rubber Hand Illusion in Reality, Virtual Reality, and Mixed Reality , 2006, PRESENCE: Teleoperators and Virtual Environments.

[20]  D. Burin,et al.  That’s my hand! Therefore, that’s my willed action: How body ownership acts upon conscious awareness of willed actions , 2017, Cognition.

[21]  Keisuke Okumura,et al.  Investigating the Hand Ownership Illusion With Two Views Merged in , 2020, Frontiers in Robotics and AI.

[22]  M. Pyasik,et al.  On the relation between body ownership and sense of agency: A link at the level of sensory-related signals. , 2018, Acta psychologica.

[23]  N. Hon,et al.  Preoccupied minds feel less control: Sense of agency is modulated by cognitive load , 2013, Consciousness and Cognition.

[24]  Stephen W. Song,et al.  Enhancement of perceived body ownership in virtual reality-based teleoperation may backfire in the execution of high-risk tasks , 2020, Computers in Human Behavior.

[25]  Nina Rosa,et al.  A Model for Virtual Hand Ownership in Augmented Reality , 2020, 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW).

[26]  S. M. Aglioti,et al.  Mere observation of body discontinuity affects perceived ownership and vicarious agency over a virtual hand , 2015, Experimental Brain Research.

[27]  Michael Potter Snow,et al.  Charting presence in virtual environments and its effects on performance , 1998 .

[28]  Marc Erich Latoschik,et al.  The Impact of Avatar Personalization and Immersion on Virtual Body Ownership, Presence, and Emotional Response , 2018, IEEE Transactions on Visualization and Computer Graphics.

[29]  A. Yamashita,et al.  The Sense of Agency during Continuous Action: Performance Is More Important than Action-Feedback Association , 2015, PloS one.

[30]  P. Verschure,et al.  Self Beyond the Body: Action-Driven and Task-Relevant Purely Distal Cues Modulate Performance and Body Ownership , 2019, Front. Hum. Neurosci..

[31]  P. Haggard,et al.  Having a body versus moving your body: Neural signatures of agency and body-ownership , 2010, Neuropsychologia.

[32]  H. Critchley,et al.  Multisensory integration across exteroceptive and interoceptive domains modulates self-experience in the rubber-hand illusion , 2013, Neuropsychologia.

[33]  M. Pyasik,et al.  Visual appearance of the virtual hand affects embodiment in the virtual hand illusion , 2020, Scientific Reports.

[34]  Alexander Toet,et al.  Toward Enhanced Teleoperation Through Embodiment , 2020, Frontiers in Robotics and AI.

[35]  H. Ehrsson,et al.  Moving a Rubber Hand that Feels Like Your Own: A Dissociation of Ownership and Agency , 2012, Front. Hum. Neurosci..