The reality of virtual reality

Virtual reality (VR) has become a popular tool for investigating human behavior and brain functions. Nevertheless, it is unclear whether VR constitutes an actual form of reality or is more like an advanced simulation. Determining the nature of VR has been mostly achieved by self-reported presence measurements, defined as the feeling of being submerged in the experience. However, subjective measurements might be prone to bias and, most importantly, do not allow for a comparison with real-life experiences. Here, we show that real-life and VR height exposures using 3D-360° videos are mostly indistinguishable on a psychophysiological level (EEG and HRV), while both differ from a conventional 2D laboratory setting. Using a fire truck, three groups of participants experienced a real-life (N = 25), a virtual (N = 24), or a 2D laboratory (N = 25) height exposure. Behavioral and psychophysiological results suggest that identical exogenous and endogenous cognitive as well as emotional mechanisms are deployed to process the real-life and virtual experience. Specifically, alpha- and theta-band oscillations in line with heart rate variability, indexing vigilance, and anxiety were barely indistinguishable between those two conditions, while they differed significantly from the laboratory setup. Sensory processing, as reflected by beta-band oscillations, exhibits a different pattern for all conditions, indicating further room for improving VR on a haptic level. In conclusion, the study shows that contemporary photorealistic VR setups are technologically capable of mimicking reality, thus paving the way for the investigation of real-world cognitive and emotional processes under controlled laboratory conditions. For a video summary, see https://youtu.be/fPIrIajpfiA.

[1]  Dooley J. Murphy,et al.  What Do We Mean When We Say “Presence”? , 2020, PRESENCE: Virtual and Augmented Reality.

[2]  J. Baumeister,et al.  Continuous table tennis is associated with processing in frontal brain areas: an EEG approach , 2022, Experimental Brain Research.

[3]  Thomas Gruber,et al.  Authentic Fear Responses in Virtual Reality: A Mobile EEG Study on Affective, Behavioral and Electrophysiological Correlates of Fear , 2021, Frontiers in Virtual Reality.

[4]  Phillip M. Alday,et al.  EEG and behavioral correlates of attentional processing while walking and navigating naturalistic environments , 2021, Scientific Reports.

[5]  T. Gruber,et al.  Library for universal virtual reality experiments (luVRe): A standardized immersive 3D/360° picture and video database for VR based research , 2021, Current Psychology.

[6]  Mark S. Hancock,et al.  You’re Making Me Sick: A Systematic Review of How Virtual Reality Research Considers Gender & Cybersickness , 2021, CHI.

[7]  Daniel M. Shafer,et al.  The effects of interaction fidelity on game experience in virtual reality. , 2021 .

[8]  H. Schupp,et al.  Emotion and Brain Oscillations: High Arousal is Associated with Decreases in Alpha- and Lower Beta-Band Power. , 2020, Cerebral cortex.

[9]  T. Hartig,et al.  An Actual Natural Setting Improves Mood Better Than Its Virtual Counterpart: A Meta-Analysis of Experimental Data , 2020, Frontiers in Psychology.

[10]  Roman Osinsky,et al.  Aiming at ecological validity—Midfrontal theta oscillations in a toy gun shooting task , 2020, The European journal of neuroscience.

[11]  T. Gruber,et al.  Virtual reality experiences promote autobiographical retrieval mechanisms: Electrophysiological correlates of laboratory and virtual experiences , 2020, Psychological Research.

[12]  O. Güntürkün,et al.  Investigating real-life emotions in romantic couples: a mobile EEG study , 2020, bioRxiv.

[13]  D. Mograbi,et al.  Mood Induction Using Virtual Reality: a Systematic Review of Recent Findings , 2020, Journal of Technology in Behavioral Science.

[14]  D. Keim,et al.  Virtual Reality Potentiates Emotion and Task Effects of Alpha/Beta Brain Oscillations , 2020, Brain sciences.

[15]  Anna Felnhofer,et al.  Does virtual reality help to cut the Gordian knot between ecological validity and experimental control? , 2020 .

[16]  Thomas Gruber,et al.  Sustained inattentional blindness in virtual reality and under conventional laboratory conditions , 2020, Virtual Reality.

[17]  C. Perez,et al.  Invisible Women: Exposing Data Bias in a World Designed for Men , 2020 .

[18]  Mel Slater,et al.  The Ethics of Realism in Virtual and Augmented Reality , 2020, Frontiers in Virtual Reality.

[19]  O. Güntürkün,et al.  Using Mobile EEG to Investigate Alpha and Beta Asymmetries During Hand and Foot Use , 2020, Frontiers in Neuroscience.

[20]  G. Hajcak,et al.  Acute Stress Reduces Reward-Related Neural Activity: Evidence from the Reward Positivity , 2020, bioRxiv.

[21]  Umer Asghar Chattha,et al.  Motion Sickness in Virtual Reality: An Empirical Evaluation , 2020, IEEE Access.

[22]  Karsten Morisse,et al.  Impact of Virtual Embodiment on the Perception of Virtual Heights , 2020 .

[23]  G. Riva,et al.  Virtual Reality , 2020, The Palgrave Encyclopedia of the Possible.

[24]  J. Galen Buckwalter,et al.  Heart Rate Variability as an Index of Resilience. , 2020, Military medicine.

[25]  S. Mallam,et al.  Incidental Memory Recall in Virtual Reality: An Empirical Investigation , 2019, Proceedings of the Human Factors and Ergonomics Society Annual Meeting.

[26]  J. Collett,et al.  Virtual skills training: the role of presence and agency , 2019, Heliyon.

[27]  Walter J. Greenleaf,et al.  Using Virtual Reality Exposure Therapy to Enhance Treatment of Anxiety Disorders: Identifying Areas of Clinical Adoption and Potential Obstacles , 2019, Front. Psychiatry.

[28]  T. Gruber,et al.  Behavioral realism and lifelike psychophysiological responses in virtual reality by the example of a height exposure , 2019, Psychological Research.

[29]  S. Smith,et al.  Virtual reality in episodic memory research: A review , 2019, Psychonomic Bulletin & Review.

[30]  Thomas Gruber,et al.  Experiences in Virtual Reality: a Window to Autobiographical Memory , 2019 .

[31]  T. Gruber,et al.  Experiences in virtual reality entail different processes of retrieval as opposed to conventional laboratory settings: A study on human memory , 2019, Current Psychology.

[32]  Andrea Gaggioli,et al.  When Virtual Feels Real: Comparing Emotional Responses and Presence in Virtual and Natural Environments , 2019, Cyberpsychology Behav. Soc. Netw..

[33]  Tonio Ball,et al.  Brain Activity in Virtual Reality: Assessing Signal Quality of High-Resolution EEG While Using Head-Mounted Displays , 2019, 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[34]  P. Pauli,et al.  Causal Interactive Links Between Presence and Fear in Virtual Reality Height Exposure , 2019, Front. Psychol..

[35]  Arno Villringer,et al.  Decoding Subjective Emotional Arousal during a Naturalistic VR Experience from EEG Using LSTMs , 2018, 2018 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR).

[36]  Julie Prescott,et al.  Implementations of Virtual Reality for Anxiety-Related Disorders: Systematic Review , 2018, JMIR serious games.

[37]  G. Riva,et al.  The Past, Present, and Future of Virtual and Augmented Reality Research: A Network and Cluster Analysis of the Literature , 2018, Front. Psychol..

[38]  Silvia Serino,et al.  New Trends in Episodic Memory Assessment: Immersive 360° Ecological Videos , 2018, Front. Psychol..

[39]  Jacqueline M. Fulvio,et al.  Sensitivity to Sensory Cues Predicts Motion Sickness in Virtual Reality , 2018, Journal of Vision.

[40]  Bobby Bodenheimer,et al.  Perception of height in virtual reality: a study of climbing stairs , 2018, SAP.

[41]  M. Slater Immersion and the illusion of presence in virtual reality. , 2018, British journal of psychology.

[42]  Daniel P Ferris,et al.  Effects of virtual reality high heights exposure during beam-walking on physiological stress and cognitive loading , 2018, PloS one.

[43]  Paul Pauli,et al.  Height Simulation in a Virtual Reality CAVE System: Validity of Fear Responses and Effects of an Immersion Manipulation , 2018, Front. Hum. Neurosci..

[44]  C. Plaisant,et al.  Virtual memory palaces: immersion aids recall , 2018, Virtual Reality.

[45]  Arno Villringer,et al.  Using EEG to Decode Subjective Levels of Emotional Arousal During an Immersive VR Roller Coaster Ride , 2018, 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[46]  Andrea Gaggioli,et al.  Designing Awe in Virtual Reality: An Experimental Study , 2018, Front. Psychol..

[47]  Marko Horvat,et al.  Assessing emotional responses induced in virtual reality using a consumer EEG headset: A preliminary report , 2018, 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO).

[48]  T. Lonsdorf,et al.  An elevated plus-maze in mixed reality for studying human anxiety-related behavior , 2017, BMC Biology.

[49]  J. L. Higuera-Trujillo,et al.  Psychological and physiological human responses to simulated and real environments: A comparison between Photographs, 360° Panoramas, and Virtual Reality. , 2017, Applied ergonomics.

[50]  F. Shaffer,et al.  An Overview of Heart Rate Variability Metrics and Norms , 2017, Front. Public Health.

[51]  Daniel M. Johnson,et al.  Improved Memory Elicitation in Virtual Reality: New Experimental Results and Insights , 2017, INTERACT.

[52]  Michael L. Wilson,et al.  Absence of Gender Differences in Actual Induced HMD Motion Sickness vs. Pretrial Susceptibility Ratings , 2017 .

[53]  Antonio Egidio Nardi,et al.  Virtual Reality as a Promising Strategy in the Assessment and Treatment of Bulimia Nervosa and Binge Eating Disorder: A Systematic Review , 2017, Behavioral sciences.

[54]  Jih-Hsuan Tammy Lin,et al.  Fear in virtual reality (VR): Fear elements, coping reactions, immediate and next-day fright responses toward a survival horror zombie virtual reality game , 2017, Comput. Hum. Behav..

[55]  Daniel P. Ferris,et al.  Independent Component Analysis and Source Localization on Mobile EEG Data Can Identify Increased Levels of Acute Stress , 2017, Front. Hum. Neurosci..

[56]  D. Lakens Equivalence Tests , 2017, Social psychological and personality science.

[57]  Luca Chittaro,et al.  Mortality salience in virtual reality experiences and its effects on users' attitudes towards risk , 2017, Int. J. Hum. Comput. Stud..

[58]  G. Riva,et al.  Effectiveness of Immersive Videos in Inducing Awe: An Experimental Study , 2017, Scientific Reports.

[59]  Stéphane Bouchard,et al.  Virtual reality compared with in vivo exposure in the treatment of social anxiety disorder: A three-arm randomised controlled trial , 2017, British Journal of Psychiatry.

[60]  Sang Min Lee,et al.  Electroencephalographic Correlates of Suicidal Ideation in the Theta Band , 2017, Clinical EEG and neuroscience.

[61]  Michael Rose,et al.  Differential effects of ongoing EEG beta and theta power on memory formation , 2017, PloS one.

[62]  Mel Slater,et al.  Enhancing Our Lives with Immersive Virtual Reality , 2016, Front. Robot. AI.

[63]  Stefania Serafin,et al.  Immersion revisited: A review of existing definitions of immersion and their relation to different theories of presence , 2016 .

[64]  T. Gruber,et al.  Event-related frontal alpha asymmetries: electrophysiological correlates of approach motivation , 2016, Experimental Brain Research.

[65]  T. Parsons Virtual Reality for Enhanced Ecological Validity and Experimental Control in the Clinical, Affective and Social Neurosciences , 2015, Front. Hum. Neurosci..

[66]  Selin Aviyente,et al.  Sending mixed signals: worry is associated with enhanced initial error processing but reduced call for subsequent cognitive control. , 2015, Social cognitive and affective neuroscience.

[67]  J. Bhattacharya,et al.  Aroused with heart: Modulation of heartbeat evoked potential by arousal induction and its oscillatory correlates , 2015, Scientific Reports.

[68]  Helmut Hlavacs,et al.  Is virtual reality emotionally arousing? Investigating five emotion inducing virtual park scenarios , 2015, Int. J. Hum. Comput. Stud..

[69]  J. Woo,et al.  Reliability of ultra-short-term analysis as a surrogate of standard 5-min analysis of heart rate variability. , 2015, Telemedicine journal and e-health : the official journal of the American Telemedicine Association.

[70]  Rossana Castaldo,et al.  Acute mental stress assessment via short term HRV analysis in healthy adults: A systematic review with meta-analysis , 2015, Biomed. Signal Process. Control..

[71]  Henrik M. Peperkorn,et al.  The impact of perception and presence on emotional reactions: a review of research in virtual reality , 2015, Front. Psychol..

[72]  Nadia Bianchi-Berthouze,et al.  The brain’s response to pleasant touch: an EEG investigation of tactile caressing , 2014, Front. Hum. Neurosci..

[73]  Ole Jensen,et al.  Different roles of alpha and beta band oscillations in anticipatory sensorimotor gating , 2014, Front. Hum. Neurosci..

[74]  Martin Grunwald,et al.  EEG changes caused by spontaneous facial self-touch may represent emotion regulating processes and working memory maintenance , 2014, Brain Research.

[75]  Bart Verkuil,et al.  EEG theta/beta ratio as a potential biomarker for attentional control and resilience against deleterious effects of stress on attention , 2013, Cognitive, Affective, & Behavioral Neuroscience.

[76]  Andero Uusberg,et al.  Using distraction to regulate emotion: insights from EEG theta dynamics. , 2014, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[77]  Michael X Cohen,et al.  Analyzing Neural Time Series Data: Theory and Practice , 2014 .

[78]  Gregor Leicht,et al.  Emotion regulation by cognitive reappraisal — The role of frontal theta oscillations , 2013, NeuroImage.

[79]  Joanne Cantor,et al.  Why Horror Doesn’t Die: The Enduring and Paradoxical Effects of Frightening Entertainment , 2013 .

[80]  L. Mulder,et al.  The utility of low frequency heart rate variability as an index of sympathetic cardiac tone: a review with emphasis on a reanalysis of previous studies. , 2013, Psychophysiology.

[81]  Jaakko Erkkilä,et al.  Validity and reliability of electroencephalographic frontal alpha asymmetry and frontal midline theta as biomarkers for depression. , 2013, Scandinavian journal of psychology.

[82]  Nelson J. Trujillo-Barreto,et al.  Successful memory encoding is associated with increased cross-frequency coupling between frontal theta and posterior gamma oscillations in human scalp-recorded EEG , 2013, NeuroImage.

[83]  Nicola Vitiello,et al.  Synthetic and Bio-Artificial Tactile Sensing: A Review , 2013, Sensors.

[84]  W. Klimesch Alpha-band oscillations, attention, and controlled access to stored information , 2012, Trends in Cognitive Sciences.

[85]  Louis Renoult,et al.  Personal semantics: at the crossroads of semantic and episodic memory , 2012, Trends in Cognitive Sciences.

[86]  R. Chellali,et al.  Tactile exploration of virtual objects for blind and sighted people: the role of beta 1 EEG band in sensory substitution and supramodal mental mapping. , 2012, Journal of neurophysiology.

[87]  C. Botella,et al.  Virtual reality exposure therapy in anxiety disorders: a quantitative meta‐analysis , 2012, Depression and anxiety.

[88]  Anthony Singhal,et al.  Theta oscillations reflect a putative neural mechanism for human sensorimotor integration. , 2012, Journal of neurophysiology.

[89]  Corey J. Bohil,et al.  Virtual reality in neuroscience research and therapy , 2011, Nature Reviews Neuroscience.

[90]  Stephanie E. Moser,et al.  Feeling good: autonomic nervous system responding in five positive emotions. , 2011, Emotion.

[91]  R. B. Reilly,et al.  FASTER: Fully Automated Statistical Thresholding for EEG artifact Rejection , 2010, Journal of Neuroscience Methods.

[92]  A. Kleinschmidt,et al.  Intrinsic Connectivity Networks, Alpha Oscillations, and Tonic Alertness: A Simultaneous Electroencephalography/Functional Magnetic Resonance Imaging Study , 2010, The Journal of Neuroscience.

[93]  G. Riva,et al.  Assessment of the emotional responses produced by exposure to real food, virtual food and photographs of food in patients affected by eating disorders , 2010, Annals of general psychiatry.

[94]  B. Olatunji,et al.  Emotion Regulation and the Anxiety Disorders: An Integrative Review , 2010, Journal of psychopathology and behavioral assessment.

[95]  Mel Slater,et al.  Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[96]  C. McLean,et al.  Brave men and timid women? A review of the gender differences in fear and anxiety. , 2009, Clinical psychology review.

[97]  S. Huffel,et al.  Influence of Mental Stress on Heart Rate and Heart Rate Variability , 2009 .

[98]  Matthias M. Müller,et al.  Induced Electroencephalogram Oscillations during Source Memory: Familiarity is Reflected in the Gamma Band, Recollection in the Theta Band , 2008, Journal of Cognitive Neuroscience.

[99]  Myeong Gi Jeong,et al.  Ultra Short Term Analysis of Heart Rate Variability for Monitoring Mental Stress in Mobile Settings , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[100]  Mariano Alcañiz Raya,et al.  Affective Interactions Using Virtual Reality: The Link between Presence and Emotions , 2007, Cyberpsychology Behav. Soc. Netw..

[101]  John J. Foxe,et al.  Optimal sustained attention is linked to the spectral content of background EEG activity: greater ongoing tonic alpha (∼10 Hz) power supports successful phasic goal activation , 2007 .

[102]  Hongwei Hsiao,et al.  Height Effects in Real and Virtual Environments , 2005, Hum. Factors.

[103]  Benoît G. Bardy,et al.  Postural Responses to Two Technologies for Generating Optical Flow , 2004, Presence: Teleoperators & Virtual Environments.

[104]  A. Savostyanov,et al.  Alpha oscillations as a correlate of trait anxiety. , 2004, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[105]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[106]  M. Krijn,et al.  Treatment of acrophobia in virtual reality: the role of immersion and presence. , 2004, Behaviour research and therapy.

[107]  Manuel Schabus,et al.  Theta coupling in the human electroencephalogram during a working memory task , 2004, Neuroscience Letters.

[108]  Thomas A. Stoffregen,et al.  On the nature and evaluation of fidelity in virtual environments: Applications, Implications, and Human Performance , 2019 .

[109]  David C. Rubin,et al.  The Neuropsychology of Autobiographical Memory , 2003, Cortex.

[110]  F. Varela,et al.  Neuromagnetic imaging of cortical oscillations accompanying tactile stimulation. , 2003, Brain research. Cognitive brain research.

[111]  G. Riva,et al.  Being There: Concepts, Effects and Measurements of User Presence in Synthetic Environments , 2003 .

[112]  O. Jensen,et al.  Frontal theta activity in humans increases with memory load in a working memory task , 2002, The European journal of neuroscience.

[113]  Crystal L. Hoyt,et al.  Immersive Virtual Environment Technology as a Methodological Tool for Social Psychology , 2002 .

[114]  Dong Pyo Jang,et al.  Attention Enhancement System using virtual reality and EEG biofeedback , 2002, Proceedings IEEE Virtual Reality 2002.

[115]  Y. Benjamini,et al.  THE CONTROL OF THE FALSE DISCOVERY RATE IN MULTIPLE TESTING UNDER DEPENDENCY , 2001 .

[116]  Holger Regenbrecht,et al.  The Experience of Presence: Factor Analytic Insights , 2001, Presence: Teleoperators & Virtual Environments.

[117]  Mel Slater,et al.  Using Presence Questionnaires in Reality , 2000, Presence: Teleoperators & Virtual Environments.

[118]  Wijnand A. IJsselsteijn,et al.  Presence: concept, determinants, and measurement , 2000, Electronic Imaging.

[119]  Wijnand A. IJsselsteijn,et al.  Using Behavioral Realism to Estimate Presence: A Study of the Utility of Postural Responses to Motion Stimuli , 2000, Presence: Teleoperators & Virtual Environments.

[120]  Yasushi Mizuki,et al.  Appearance of Frontal Midline Theta Activity in Patients with Generalized Anxiety Disorder , 2000, Neuropsychobiology.

[121]  W. Klimesch EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis , 1999, Brain Research Reviews.

[122]  G Arnold,et al.  Effect of respiration rate on short-term heart rate variability , 1999 .

[123]  M. Zuckerman Item revisions in the Sensation Seeking Scale Form V (SSS-V) , 1996 .

[124]  Carl-Walter Kohlmann,et al.  Untersuchungen mit einer deutschen Version der "Positive and Negative Affect Schedule" (PANAS). , 1996 .

[125]  Lawrence W. Stark,et al.  The Effects of Pictorial Realism, Delay of Visual Feedback, and Observer Interactivity on the Subjective Sense of Presence , 1996, Presence: Teleoperators & Virtual Environments.

[126]  Rob Kooper,et al.  Virtual Environments for Treating the Fear of Heights , 1995, Computer.

[127]  James S. Williford,et al.  Virtual Reality Graded Exposure in the Treatment of Acrophobia: A Case Report , 1995 .

[128]  Mel Slater,et al.  Depth of Presence in Virtual Environments , 1994, Presence: Teleoperators & Virtual Environments.

[129]  Thomas B. Sheridan,et al.  Musings on Telepresence and Virtual Presence , 1992, Presence: Teleoperators & Virtual Environments.

[130]  D. Lougson,et al.  Acute. , 2020, The Manchester medical gazette.

[131]  D. C. Cohen,et al.  Comparison of self-report and overt-behavioral procedures for assessing acrophobia. , 1977 .

[132]  E. Gellhorn The emotions and the ergotropic and trophotropic systems , 1970, Psychologische Forschung.

[133]  D. R. Davies,et al.  SKIN-CONDUCTANCE, ALPHA-ACTIVITY, AND VIGILANCE. , 1965, The American journal of psychology.