Psychophysical Effects of Experiencing Burning Hands in Augmented Reality

Can interactive Augmented Reality (AR) experiences induce involuntary sensations through additional modalities? In this paper we report on our AR experience that enables users to see and hear their own hands burning while looking through a Video See-Through Head-Mounted Display (VST-HMD). In an exploratory study (n = 12, within-subject design), we investigated whether this will lead to an involuntary heat sensation based on visual and auditory stimuli. A think-aloud-protocol and an AR presence questionnaire indicated that six out of twelve participants experienced an involuntary heat sensation on their hands. Despite no significant change of perceived anxiety, we found a significant increase in skin conductance during the experiment for all participants; participants who reported an involuntary heat sensation had higher skin conductance responses than participants who did not report a heat sensation. Our results support our initial hypothesis as we found evidence of cross-modal audiovisual-to-thermal transfers. This is an example of virtual synaesthesia, a sensation occurring when single-modal (or multi-modal) stimulus sets off the simultaneous sensation over other senses—involuntarily and automatically. We believe that our results contribute to the scientific understanding of AR induced synaesthesia as well as inform practical applications.

[1]  A. Rogowska,et al.  Categorization of Synaesthesia , 2011 .

[2]  Mel Slater,et al.  A Virtual Presence Counter , 2000, Presence: Teleoperators & Virtual Environments.

[3]  Blair MacIntyre,et al.  Experiences with an AR evaluation test bed: Presence, performance, and physiological measurement , 2010, 2010 IEEE International Symposium on Mixed and Augmented Reality.

[4]  Ana L. N. Fred,et al.  BITalino - A Multimodal Platform for Physiological Computing , 2013, ICINCO.

[5]  Luca Citi,et al.  cvxEDA: A Convex Optimization Approach to Electrodermal Activity Processing , 2016, IEEE Transactions on Biomedical Engineering.

[6]  Bartholomäus Wissmath,et al.  Immersion in Mediated Environments: The Role of Personality Traits , 2009, Cyberpsychology Behav. Soc. Netw..

[7]  Steven K. Feiner,et al.  Breaking the Barriers to True Augmented Reality , 2015, ArXiv.

[8]  Jim Blascovich,et al.  Subjective, physiological, and behavioral effects of threat and challenge appraisal. , 1993 .

[9]  Frank Biocca,et al.  Visual Touch in Virtual Environments: An Exploratory Study of Presence, Multimodal Interfaces, and Cross-Modal Sensory Illusions , 2001, Presence: Teleoperators & Virtual Environments.

[10]  Arindam Dey,et al.  Burnar: Involuntary heat sensations in augmented reality , 2013, 2013 IEEE Virtual Reality (VR).

[11]  Abderrahmane Kheddar,et al.  Pseudo-haptic feedback: can isometric input devices simulate force feedback? , 2000, Proceedings IEEE Virtual Reality 2000 (Cat. No.00CB37048).

[12]  E. Gallagher,et al.  No proof of a difference is not equivalent to proof of no difference. , 1994, The Journal of emergency medicine.

[13]  Shih-Ching Yeh,et al.  Effects of Virtual Reality and Augmented Reality on Induced Anxiety. , 2018, IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[14]  P. Stone,et al.  Use of skin conductance changes during mental stress testing as an index of autonomic arousal in cardiovascular research. , 1994, American heart journal.

[15]  C. Spielberger.  State‐Trait Anxiety Inventory , 2010 .

[16]  Carlos M. Coelho,et al.  Media Presence and Inner Presence: The Sense of Presence in Virtual Reality Technologies , 2006 .

[17]  Mel Slater,et al.  Human Tails: Ownership and Control of Extended Humanoid Avatars , 2013, IEEE Transactions on Visualization and Computer Graphics.

[18]  Hunter G. Hoffman,et al.  Modulation of thermal pain-related brain activity with virtual reality: evidence from fMRI , 2004, Neuroreport.

[19]  Doug L. James,et al.  Animating fire with sound , 2011, SIGGRAPH 2011.

[20]  Takuji Narumi,et al.  Augmented reality flavors: gustatory display based on edible marker and cross-modal interaction , 2011, CHI.

[21]  M. Krijn,et al.  Virtual reality exposure therapy of anxiety disorders: a review. , 2004, Clinical psychology review.

[22]  Holger Regenbrecht,et al.  Measuring Cognitive Load and Insight: A Methodology Exemplified in a Virtual Reality Learning Context , 2019, 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[23]  Rosa María Baños,et al.  Using augmented reality to treat phobias , 2005, IEEE Computer Graphics and Applications.

[24]  M. A. Rowe,et al.  Guide for Analysing Electrodermal Activity & Skin Conductance Responses for Psychological Experiments , 2013 .

[25]  Dawn Iacobucci,et al.  Toward a More Nuanced Understanding of the Statistical Properties of a Median Split , 2015 .

[26]  Maria V. Sanchez-Vives,et al.  From presence to consciousness through virtual reality , 2005, Nature Reviews Neuroscience.

[27]  Christine Youngblut.  What a Decade of Experiments Reveals about Factors that Influence the Sense of Presence: Latest Findings , 2006 .

[28]  R. Benjamin Knapp,et al.  Affective feedback in a virtual reality based intelligent supermarket , 2017, UbiComp/ISWC Adjunct.

[29]  Helmut Hlavacs,et al.  Afraid to Be There? Evaluating the Relation Between Presence, Self-Reported Anxiety, and Heart Rate in a Virtual Public Speaking Task , 2014, Cyberpsychology Behav. Soc. Netw..

[30]  Anatole Lécuyer,et al.  Influence of Personality Traits and Body Awareness on the Sense of Embodiment in Virtual Reality , 2019, 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[31]  Mel Slater,et al.  It feels real: physiological responses to a stressful virtual reality environment and its impact on working memory , 2019, Journal of psychopharmacology.

[32]  Larry F. Hodges,et al.  Can Audio Enhance Visual Perception and Performance in a Virtual Environment? , 1999 .

[33]  Frank H. Durgin,et al.  Rubber Hands Feel the Touch of Light , 2007, Psychological science.

[34]  J. Tomaka,et al.  Assertiveness predicts threat and challenge reactions to potential stress among women. , 1999, Journal of personality and social psychology.

[35]  R. Cytowič.  Synesthesia: A Union of the Senses , 1989 .

[36]  Takuji Narumi,et al.  Visual-olfactory display using olfactory sensory map , 2010, 2010 IEEE Virtual Reality Conference (VR).

[37]  Olivier Martin,et al.  HEMP - hand-displacement-based pseudo-haptics: A study of a force field application and a behavioural analysis , 2009, Int. J. Hum. Comput. Stud..

[38]  Ronald Azuma,et al.  Recent Advances in Augmented Reality , 2001, IEEE Computer Graphics and Applications.

[39]  Helene S. Wallach,et al.  The Impact of Personality Traits on the Experience of Presence , 2009, 2009 Virtual Rehabilitation International Conference.