Assessing Engagement during Rescue Operation Simulated in Virtual Reality: A Psychophysiological Study

ABSTRACT The goal of this study was to investigate the patterns of engagement among professional firefighters during a rescue operation challenge simulated in a virtual reality (VR). The simulator offers a training that would otherwise be impossible or very difficult to arrange in the real world, here a mass-casualty incident. We measured engagement with cardiovascular reactivity as well as subjective perceptions of workload. We found that both a VR rescue challenge and a VR control condition lead to engagement evident in the decrease in parasympathetic activation from baseline (measured as high-frequency heart rate variability). However, the rescue operation leads to a stronger increase in sympathetic activity (shorter pre-ejection period and RZ-interval) than the control condition. Furthermore, the subjective workload ratings corroborate the results from the objective engagement indices. These results demonstrate that it is possible to create a virtual environment that elicits engagement among professional rescuers.

[1]  J. Fahrenberg,et al.  Methodological guidelines for impedance cardiography. , 1990, Psychophysiology.

[2]  G. Berntson,et al.  Handbook of Psychophysiology: Cardiovascular Psychophysiology , 2007 .

[3]  J. Cacioppo,et al.  Autonomic determinism: the modes of autonomic control, the doctrine of autonomic space, and the laws of autonomic constraint. , 1991, Psychological review.

[4]  V Louhevaara,et al.  Are the job demands on physical work capacity equal for young and aging firefighters? , 1994, Journal of occupational medicine. : official publication of the Industrial Medical Association.

[5]  John R. Wilson,et al.  Virtual Reality-Induced Symptoms and Effects (VRISE) , 1999, Presence: Teleoperators & Virtual Environments.

[6]  Jurriaan Bos,et al.  Risk of health complaints and disabilities among Dutch firefighters , 2004, International archives of occupational and environmental health.

[7]  T L Guidotti,et al.  Human factors in firefighting: ergonomic-, cardiopulmonary-, and psychogenic stress-related issues , 1992, International archives of occupational and environmental health.

[8]  Bogdan Gabrys,et al.  An overview of self-adaptive technologies within virtual reality training , 2016, Comput. Sci. Rev..

[9]  Roger E. Beaty,et al.  Creative motivation: Creative achievement predicts cardiac autonomic markers of effort during divergent thinking , 2014, Biological Psychology.

[10]  Michael Richter,et al.  A Closer Look Into the Multi‐Layer Structure of Motivational Intensity Theory , 2013 .

[11]  A. Furnham,et al.  The good, the bad and the mad: Response bias in self-report measures , 1982 .

[12]  R. Baumeister,et al.  Self-control as limited resource: regulatory depletion patterns. , 1998, Journal of personality and social psychology.

[13]  M. Richter.  Pay attention to your manipulation checks! Reward impact on cardiac reactivity is moderated by task context , 2010, Biological Psychology.

[14]  Albert A. Rizzo,et al.  Psychophysiology to Assess Impact of Varying Levels of Simulation Fidelity in a Threat Environment , 2012, Adv. Hum. Comput. Interact..

[15]  Carrie Heater,et al.  Being There: The Subjective Experience of Presence , 1992, Presence: Teleoperators & Virtual Environments.

[16]  Deanna M McDeavitt.  Searching for Effective Training Solutions for Firefighting: The Analysis of Emergency Responses and Line of Duty Death Reports for Low Frequency, High Risk Events , 2017 .

[17]  L. Rowell.  Human Cardiovascular Control , 1993 .

[18]  J. Harrald.  Agility and Discipline: Critical Success Factors for Disaster Response , 2006 .

[19]  Y. Yamamoto,et al.  Instrument for controlling drowsiness using galvanic skin reflex , 1992, Medical and Biological Engineering and Computing.

[20]  P. Raven,et al.  Autonomic neural control of heart rate during dynamic exercise: revisited , 2014, The Journal of physiology.

[21]  A. Bakker,et al.  Student engagement and performance: A weekly diary study on the role of openness , 2014, Motivation and Emotion.

[22]  B. Mineo,et al.  Engagement with Electronic Screen Media Among Students with Autism Spectrum Disorders , 2009, Journal of autism and developmental disorders.

[23]  Willis J. Tompkins,et al.  A Real-Time QRS Detection Algorithm , 1985, IEEE Transactions on Biomedical Engineering.

[24]  H. Nagaraja,et al.  Heart rate variability: origins, methods, and interpretive caveats. , 1997, Psychophysiology.

[25]  Leandro Barbosa S. Gatto,et al.  Virtual simulation of a nuclear power plant's control room as a tool for ergonomic evaluation , 2013 .

[26]  J. Brehm,et al.  The intensity of motivation. , 1989, Annual review of psychology.

[27]  Delroy L. Paulhus,et al.  Egoistic and Moralistic Biases in Self-Perception: The Interplay of Self-Deceptive Styles With Basic Traits and Motives , 1998 .

[28]  P. Silvia,et al.  Evaluating the Distorting Effects of Inattentive Responding and Social Desirability on Self-Report Scales in Creativity and the Arts. , 2017 .

[29]  D L Smith,et al.  Effect of strenuous live-fire drills on cardiovascular and psychological responses of recruit firefighters , 2001, Ergonomics.

[30]  Anna Lisa Martin-Niedecken,et al.  Go with the Dual Flow: Evaluating the Psychophysiological Adaptive Fitness Game Environment "Plunder Planet" , 2017, JCSG.

[31]  B. Bone,et al.  Selected physiological and psychobiological responses to physical activity in different configurations of firefighting gear , 1995 .

[32]  K. Vinod,et al.  Sampling frequency of the RR interval time series for spectral analysis of heart rate variability , 2004, Journal of medical engineering & technology.

[33]  Joseph Psotka,et al.  Immersive training systems: Virtual reality and education and training , 1995 .

[34]  L. Mulder,et al.  Introducing a baroreflex model for studying cardiovascular effects of mental workload. , 2004, Psychophysiology.

[35]  N Lippman,et al.  Comparison of methods for removal of ectopy in measurement of heart rate variability. , 1994, The American journal of physiology.

[36]  A. Pope,et al.  Biocybernetic system evaluates indices of operator engagement in automated task , 1995, Biological Psychology.

[37]  Ilona Heldal,et al.  Simulations and Serious Games for Firefighter Training: Users' Perspective , 2017, ISCRAM.

[38]  E. Erdfelder,et al.  Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses , 2009, Behavior research methods.

[39]  Arthur C. Graesser,et al.  Better to be frustrated than bored: The incidence, persistence, and impact of learners' cognitive-affective states during interactions with three different computer-based learning environments , 2010, Int. J. Hum. Comput. Stud..

[40]  Lachlan MacKinnon,et al.  ALT-C 2011 Proceedings Papers: 0151 The Development of a Rich Multimedia Training Environment for Crisis Management: Using Emotional Affect to Enhance Learning. , 2011 .

[41]  G. Breithardt,et al.  Heart rate variability: standards of measurement, physiological interpretation and clinical use. Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology. , 1996 .

[42]  Peter Jaye,et al.  Development and implementation of centralized simulation training: evaluation of feasibility, acceptability and construct validity , 2013, BJU international.

[43]  A. Malliani,et al.  Heart rate variability. Standards of measurement, physiological interpretation, and clinical use , 1996 .

[44]  M. Whitton,et al.  Review of Four Studies on the Use of Physiological Reaction as a Measure of Presence in StressfulVirtual Environments , 2005, Applied psychophysiology and biofeedback.

[45]  Mario Botsch,et al.  Training in a comprehensive everyday-like virtual reality environment compared to computerized cognitive training for patients with depression , 2018, Comput. Hum. Behav..

[46]  The Impact of Motivation and Task Difficulty on Resource Engagement: Differential Influences on Cardiovascular Responses of Young and Older Adults. , 2015, Motivation science.

[47]  B. Kapralos,et al.  Using Serious Games and Virtual Simulation for Training in the Fire Service: A Review , 2015 .

[48]  Michelle K. Smith,et al.  Active learning increases student performance in science, engineering, and mathematics , 2014, Proceedings of the National Academy of Sciences.

[49]  Erol Gelenbe,et al.  Distributed Building Evacuation Simulator for Smart Emergency Management , 2010, Comput. J..

[50]  Diet‐dependent modulation of gastro‐oesphageal vagal afferent mechanosensitivity by endogenous nitric oxide , 2014, The Journal of physiology.

[51]  Per Backlund,et al.  SIDH: A Game-Based Architecture for a Training Simulator , 2009, Int. J. Comput. Games Technol..

[52]  Thomas Kirste,et al.  Design Challenges for an Integrated Disaster Management Communication and Information System , 2002 .

[53]  Christian Timmerer,et al.  An evaluation of Heart Rate and ElectroDermal Activity as an objective QoE evaluation method for immersive virtual reality environments , 2016, 2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX).

[54]  Spss,et al.  Discovering Statistics (4th Edition ed. , 2013 .

[55]  N. Schutte,et al.  Facilitating empathy through virtual reality , 2017 .

[56]  Gilsoo Jang,et al.  Development of a Virtual Reality Training System for Live-Line Workers , 2006, Int. J. Hum. Comput. Interact..

[57]  Paul Brna,et al.  Presence and Reflection in Training: Support for Learning to Improve Quality Decision-Making Skills under Time Limitations , 2001, Cyberpsychology Behav. Soc. Netw..

[58]  Francine Malouin,et al.  A Treadmill and Motion Coupled Virtual Reality System for Gait Training Post-Stroke , 2006, Cyberpsychology Behav. Soc. Netw..

[59]  James P. Bliss,et al.  The Effectiveness of Virtual Reality for Administering Spatial Navigation Training to Firefighters , 1997, Presence: Teleoperators & Virtual Environments.

[60]  L Gamberini,et al.  Responding to a fire emergency in a virtual environment: different patterns of action for different situations. , 2003, Ergonomics.

[61]  Wolfram Boucsein,et al.  The Usability of Cardiovascular and Electrodermal Measures for Adaptive Automation , 2008 .

[62]  P. Obrist,et al.  Presidential Address, 1975. The cardiovascular-behavioral interaction--as it appears today. , 1976, Psychophysiology.

[63]  ChittaroLuca,et al.  Psychological and physiological responses to stressful situations in immersive virtual reality , 2016 .

[64]  Sharona T. Levy,et al.  Now I know how! The learning process of medication administration among nursing students with non-immersive desktop virtual reality simulation , 2017, Comput. Educ..

[65]  Ingvar Holmér,et al.  Classification of metabolic and respiratory demands in fire fighting activity with extreme workloads. , 2007, Applied ergonomics.

[66]  R. Kelsey.  Beta-adrenergic cardiovascular reactivity and adaptation to stress: The cardiac pre-ejection period as an index of effort. , 2012 .

[67]  Michael Richter,et al.  How effortful is cognitive control? Insights from a novel method measuring single-trial evoked beta-adrenergic cardiac reactivity. , 2017, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[68]  Nazneen Akhter,et al.  Computer based RR-Interval detection system with ectopy correction in HRV data , 2015, 2015 International Conference on Advances in Computing, Communications and Informatics (ICACCI).

[69]  G. Berntson,et al.  Where to B in dZ/dt. , 2007, Psychophysiology.

[70]  Chih-Cheng Lu,et al.  Signal fidelity requirements for deriving impedance cardiographie measures of cardiac function over a broad heart rate range , 1993, Biological Psychology.

[71]  Timothy D. Wilson,et al.  Telling more than we can know: Verbal reports on mental processes. , 1977 .

[72]  Gary G Berntson,et al.  Where to Q in PEP. , 2004, Psychophysiology.

[73]  Per Backlund,et al.  Sidh - a Game Based Firefighter Training Simulation , 2007, 2007 11th International Conference Information Visualization (IV '07).

[74]  Ross E. Willis,et al.  Embracing Errors in Simulation-Based Training: The Effect of Error Training on Retention and Transfer of Central Venous Catheter Skills. , 2015, Journal of surgical education.

[75]  P. Obrist.  Cardiovascular Psychophysiology: A Perspective , 1981 .

[76]  D. Ecobichon,et al.  Municipal firefighter exposure groups, time spent at fires and use of self-contained-breathing-apparatus. , 2001, American journal of industrial medicine.

[77]  David Murray,et al.  Reliability and Validity of a Simulation-based Acute Care Skills Assessment for Medical Students and Residents , 2003, Anesthesiology.

[78]  Ulrike Cress,et al.  Virtual training: Making reality work? , 2022 .

[79]  Giuseppe Riva,et al.  The Role of Immersion and Narrative in Mediated Presence: The Virtual Hospital Experience , 2011, Cyberpsychology Behav. Soc. Netw..

[80]  R. A. Thuraisingham,et al.  Preprocessing RR interval time series for heart rate variability analysis and estimates of standard deviation of RR intervals , 2006, Comput. Methods Programs Biomed..

[81]  R. Wright,et al.  Brehm's theory of motivation as a model of effort and cardiovascular response. , 1996 .

[82]  Dag Svanæs,et al.  Fidelity Considerations for Simulation-Based Usability Assessments of Mobile ICT for Hospitals , 2010, Int. J. Hum. Comput. Interact..

[83]  Jennifer A. Fredricks,et al.  School Engagement: Potential of the Concept, State of the Evidence , 2004 .

[84]  S. Knardahl.  Cardiovascular psychophysiology , 2000, Annals of medicine.

[85]  Heidi Sveistrup,et al.  Feasibility, Motivation, and Selective Motor Control: Virtual Reality Compared to Conventional Home Exercise in Children with Cerebral Palsy , 2006, Cyberpsychology Behav. Soc. Netw..

[86]  Albert Rehmann,et al.  A Handbook of Flight Simulation Fidelity Requirements for Human Factors Research. , 1995 .

[87]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[88]  Luca Chittaro,et al.  Psychological and physiological responses to stressful situations in immersive virtual reality: Differences between users who practice mindfulness meditation and controls , 2016, Comput. Hum. Behav..

[89]  S. Hamstra,et al.  Reconsidering fidelity in simulation-based training. , 2014, Academic medicine : journal of the Association of American Medical Colleges.

[90]  Joachim H. Nagel,et al.  Coherent ensemble averaging techniques for impedance cardiography , 1990, [1990] Proceedings. Third Annual IEEE Symposium on Computer-Based Medical Systems.

[91]  Chris Shaw,et al.  Firefighter command training virtual environment , 2003, TAPIA '03.

[92]  Helmut Hlavacs,et al.  Afraid to Be There? Evaluating the Relation Between Presence, Self-Reported Anxiety, and Heart Rate in a Virtual Public Speaking Task , 2014, Cyberpsychology Behav. Soc. Netw..

[93]  Atsuo Murata,et al.  Effects of Duration of Immersion in a Virtual Reality Environment on Postural Stability , 2004, Int. J. Hum. Comput. Interact..

[94]  P. Silvia,et al.  Perfectionism and Effort-Related Cardiac Activity: Do Perfectionists Try Harder? , 2016, PloS one.

[95]  R. Satava,et al.  Virtual Reality Simulation for the Operating Room: Proficiency-Based Training as a Paradigm Shift in Surgical Skills Training , 2005, Annals of surgery.

[96]  Patrick G. Kenny,et al.  Embodied Conversational Virtual Patients , 2010 .

[97]  J. Gould,et al.  Current state of virtual reality simulation in robotic surgery training: a review , 2016, Surgical Endoscopy.

[98]  Kenneth Sundaraj,et al.  A Review of Computer-Generated Simulation in the Pedagogy of Cataract Surgery Training and Assessment , 2013, Int. J. Hum. Comput. Interact..

[99]  D. Reid,et al.  The Influence of Virtual Reality Play on Children'S Motivation , 2005, Canadian journal of occupational therapy. Revue canadienne d'ergotherapie.

[100]  Robin Hunicke,et al.  AI for Dynamic Difficulty Adjustment in Games , 2004 .

[101]  Helmut Hlavacs,et al.  Salivary cortisol and cardiovascular reactivity to a public speaking task in a virtual and real-life environment , 2016, Comput. Hum. Behav..

[102]  N. Schneiderman,et al.  The reliability and specificity of delta versus residualized change as measures of cardiovascular reactivity to behavioral challenges. , 1991, Psychophysiology.

[103]  M. Frese,et al.  Effectiveness of error management training: a meta-analysis. , 2008, The Journal of applied psychology.

[104]  Chris Dede,et al.  Immersive Interfaces for Engagement and Learning , 2009, Science.

[105]  Shu-Sheng Liaw,et al.  Investigating learners' attitudes toward virtual reality learning environments: Based on a constructivist approach , 2010, Comput. Educ..