Short- and long-term learning of job interview with a serious game in virtual reality: influence of eyestrain, stereoscopy, and apparatus

Do apparatuses and eyestrain have effects on learning performances and quality of experience? Materials and Methods: 42 participants played a serious game simulating a job interview with a Samsung Gear VR Head-Mounted Display (HMD) or a computer screen. Participants were randomly assigned to 3 groups: PC, HMD biocular, and HMD stereoscopy (S3D). Participants played the game thrice. Eyestrain was assessed pre- and post-exposure with six optometric measures. Learning performances were obtained in-game. Quality of experience was measured with questionnaires. Results: eyestrain was higher with HMDs than PC based on Punctum Proximum of accommodation but similar between biocular and S3D. Knowledge gain and retention were similar with HMDs and PC based on scores and response time. All groups improved response time but without statistically significant differences between HMDs and PC. Visual discomfort difference was statistically significant between PC and HMDs (biocular and S3D). Flow difference was statistically significant between PC and HMDs (biocular and S3D), with the PC group reporting higher Flow than HMD-S3D. Conclusion: short-term learning is similar between PC and HMDs. Groups initially using HMDs continued improving during long-term learning but without statistically significant difference compared to PC. Eyestrain and visual discomfort were higher with HMDs than PC. Flow was higher with the PC group. Our results show that eyestrain does not seem to decrease learning.

[1]  A. Schütz,et al.  Comparing immersive virtual reality and powerpoint as methods for delivering safety training: Impacts on risk perception, learning, and decision making , 2019, Safety Science.

[2]  Laure Leroy,et al.  Eyestrain impacts on learning job interview with a serious game in virtual reality: a randomized double-blinded study , 2018, VRST.

[3]  N. Cowan.  What are the differences between long-term, short-term, and working memory? , 2008, Progress in brain research.

[4]  Anke Huckauf,et al.  Information Processing in Real and in Stereoscopic Environments , 2019, MuC.

[5]  Peter A. Howarth,et al.  Visual fatigue caused by viewing stereoscopic motion images: Background, theories, and observations , 2008, Displays.

[6]  Mark Mon-Williams,et al.  Natural problems for stereoscopic depth perception in virtual environments , 1995, Vision Research.

[7]  Wijnand A. IJsselsteijn,et al.  Effects of Sensory Information and Prior Experience on Direct Subjective Ratings of Presence , 1999, Presence: Teleoperators & Virtual Environments.

[8]  Imed Boughzala,et al.  A Field Application to Vet a Serious Game toward Learning Objectives , 2015, 2015 48th Hawaii International Conference on System Sciences.

[9]  IV JulianAbich,et al.  A review of the evidence for training effectiveness with virtual reality technology , 2021, Virtual Reality.

[10]  Sang Hee Kweon,et al.  A Brain Wave Research on VR (Virtual Reality) Usage: Comparison Between VR and 2D Video in EEG Measurement , 2017 .

[11]  Hee-Sun Lee,et al.  The Effect of Two-dimensional and Stereoscopic Presentation on Middle School Students’ Performance of Spatial Cognition Tasks , 2010 .

[12]  Robert J. Stone,et al.  Evolution of aerospace simulation: From immersive Virtual Reality to serious games , 2011, Proceedings of 5th International Conference on Recent Advances in Space Technologies - RAST2011.

[13]  Kyle A. Bernhardt,et al.  Symptoms of convergence and accommodative insufficiency predict engagement and cognitive fatigue during complex task performance with and without automation. , 2020, Applied ergonomics.

[14]  Benjamin S. Bloom,et al.  Taxonomy of Educational Objectives: The Classification of Educational Goals. , 1957 .

[15]  T. Franklin Waddell,et al.  Psychological and Physiological Responses to Stereoscopic 3D Presentation in Handheld Digital Gaming: Comparing the Experiences of Frequent and Infrequent Game Players , 2014, PRESENCE: Teleoperators and Virtual Environments.

[16]  Robert S Allison,et al.  Stereoscopy and the Human Visual System. , 2011, SMPTE motion imaging journal.

[17]  Luca Chittaro.  Designing Serious Games for Safety Education: “Learn to Brace” versus Traditional Pictorials for Aircraft Passengers , 2016, IEEE Transactions on Visualization and Computer Graphics.

[18]  Jerald Thomas,et al.  Level of immersion affects spatial learning in virtual environments: results of a three-condition within-subjects study with long intersession intervals , 2020, Virtual Reality.

[19]  Bruno Arnaldi,et al.  Learning procedural skills with a virtual reality simulator: An acceptability study. , 2019, Nurse education today.

[20]  AlexaMarc,et al.  Keep It Simple , 2019 .

[21]  Daniel Thalmann,et al.  An evaluation of spatial presence, social presence, and interactions with various 3D displays , 2016, CASA.

[22]  V. Johnston,et al.  Impact of accommodation, convergence and stereoacuity on perceived symptoms and surgical performance among surgeons , 2020, Surgical Endoscopy.

[23]  Bai-chuan Jiang,et al.  Models of Vergence and Accommodation-Vergence Interactions , 2002 .

[24]  Luca Chittaro,et al.  A Comparison of Procedural Safety Training in Three Conditions: Virtual Reality Headset, Smartphone, and Printed Materials , 2021, IEEE Transactions on Learning Technologies.

[25]  Lisa Rebenitsch,et al.  Estimating cybersickness from virtual reality applications , 2020, Virtual Reality.

[26]  Jie Guo,et al.  Effect of Using HMDs for One Hour on Preteens Visual Fatigue , 2018, 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct).

[27]  Gunta Krumina,et al.  Ocular performance evaluation: how prolonged near work with virtual and real 3D image modifies our visual system , 2019, APPIS.

[28]  Shu-Hsuan Chang,et al.  Do virtual reality head-mounted displays make a difference? A comparison of presence and self-efficacy between head-mounted displays and desktop computer-facilitated virtual environments , 2018, Virtual Reality.

[29]  Robert W. Lindeman,et al.  On the use of ”Active Breaks” to perform Eye Exercises for more Comfortable VR Experiences , 2020, 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW).

[30]  K. Fujii,et al.  Visualization for the analysis of fluid motion , 2005, J. Vis..

[31]  Yangshun Gu,et al.  The Effects of Consumer-grade Virtual Reality Headsets on Adult Visual Function , 2020, Seminars in ophthalmology.

[32]  Saeid Nahavandi,et al.  Using biomechanics to investigate the effect of VR on eye vergence system. , 2019, Applied ergonomics.

[33]  Jacob Rosenberg,et al.  Three-dimensional laparoscopy vs 2-dimensional laparoscopy with high-definition technology for abdominal surgery: a systematic review. , 2017, American journal of surgery.

[34]  Timothy D Wilson,et al.  Application of stereoscopic visualization on surgical skill acquisition in novices , 2013, Journal of surgical education.

[35]  P. Wouters,et al.  A meta-analysis of the cognitive and motivational effects of serious games , 2013 .

[37]  John P. McIntire,et al.  Stereoscopic 3D displays and human performance: A comprehensive review , 2014, Displays.

[38]  Hiroshi Watanabe,et al.  3D visual fatigue: Interaction between ranges of binocular disparity and interocular crosstalk , 2015, 2015 IEEE 4th Global Conference on Consumer Electronics (GCCE).

[39]  Karen Blackmore,et al.  A meta-analysis of data collection in serious games research , 2015 .

[40]  Marina Papastergiou,et al.  Digital Game-Based Learning in high school Computer Science education: Impact on educational effectiveness and student motivation , 2009, Comput. Educ..

[41]  Ruggiero Lovreglio,et al.  Comparing the effectiveness of fire extinguisher virtual reality and video training , 2020, Virtual Reality.

[42]  Yunhong Zhang,et al.  The Evaluation on Visual Fatigue and Comfort Between the VR HMD and the iPad , 2020, AHFE.

[43]  Stefano Livi,et al.  Visual discomfort while watching stereoscopic three‐dimensional movies at the cinema , 2015, Ophthalmic & physiological optics : the journal of the British College of Ophthalmic Opticians.

[44]  Shu-Sheng Liaw,et al.  Investigating learners' attitudes toward virtual reality learning environments: Based on a constructivist approach , 2010, Comput. Educ..

[45]  Jacqueline M. Fulvio,et al.  Identifying Causes of and Solutions for Cybersickness in Immersive Technology: Reformulation of a Research and Development Agenda , 2020, Int. J. Hum. Comput. Interact..

[46]  Fengfeng Ke,et al.  Effects of game-based learning in an OpenSim-supported virtual environment on mathematical performance , 2017, Interact. Learn. Environ..

[47]  Alejandro Calderón,et al.  Integrating Serious Games as Learning Resources in a Software Project Management Course: The Case of ProDec , 2017, 2017 IEEE/ACM 1st International Workshop on Software Engineering Curricula for Millennials (SECM).

[48]  Pei-Jung Wu,et al.  The comparison of accommodative response and ocular movements in viewing 3D and 2D displays , 2017, Displays.

[49]  Charles F. Bellows,et al.  Impact of Three-Dimensional Vision in Laparoscopic Training , 2007, World Journal of Surgery.

[50]  Gerhard F. Buess,et al.  3D HD versus 2D HD: surgical task efficiency in standardised phantom tasks , 2012, Surgical Endoscopy.

[51]  Jari Takatalo,et al.  User Experience in 3D Stereoscopic Games , 2011 .

[52]  David M. Hoffman,et al.  The zone of comfort: Predicting visual discomfort with stereo displays. , 2011, Journal of vision.

[53]  Luca Chittaro,et al.  Effects of Different Types of Virtual Reality Display on Presence and Learning in a Safety Training Scenario , 2018, IEEE Transactions on Visualization and Computer Graphics.

[54]  David Checa,et al.  A review of immersive virtual reality serious games to enhance learning and training , 2019, Multimedia Tools and Applications.

[55]  Nikos Papadopoulos,et al.  Full immersive virtual environment CAVETM in chemistry education , 2008, Comput. Educ..

[56]  Mark Mon-Williams,et al.  Binocular Virtual Reality Displays: When Problems Do and Don't Occur , 1998, Hum. Factors.

[57]  N. Mcbrien,et al.  Effects of a Head-Mounted Display on the Oculomotor System of Children , 2009, Optometry and vision science : official publication of the American Academy of Optometry.

[58]  Craig Stewart,et al.  Serendipitous learning & serious games: A Pilot Study , 2014, 2014 International Conference on Interactive Mobile Communication Technologies and Learning (IMCL2014).

[59]  Ji-Soo Kim.  Research on Efficient Usage of 3D Stereoscopic Technology , 2010 .

[60]  Y. Matsuura.  Aftereffect of Stereoscopic Viewing on Human Body II , 2018, Current Topics in Environmental Health and Preventive Medicine.

[61]  Influence of virtual reality on visual parameters: immersive versus non-immersive mode , 2020, BMC Ophthalmology.

[62]  Matt C. Howard,et al.  A meta-analysis of virtual reality training programs for social skill development , 2020, Comput. Educ..

[63]  Kholoud A. Bokhary,et al.  Effects of Virtual Reality Head-mounted Displays on Oculomotor Functions , 2021, International Journal of Ophthalmology & Visual Science.

[64]  Hui Fang,et al.  Effects of Dynamic Disparity on Visual Fatigue Caused by Watching 2D Videos in HMDs , 2019, IGTA.

[65]  Hyeon-Jeong Suk,et al.  Comparison of Visual Discomfort and Visual Fatigue between Head-Mounted Display and Smartphone , 2017, HVEI.

[66]  F. Okano,et al.  Repeated vergence adaptation causes the decline of visual functions in watching stereoscopic television , 2005, Journal of Display Technology.

[67]  Isabelle Hupont,et al.  How do new visual immersive systems influence gaming QoE? A use case of serious gaming with Oculus Rift , 2015, 2015 Seventh International Workshop on Quality of Multimedia Experience (QoMEX).

[68]  C. Schor,et al.  A dynamic model of cross-coupling between accommodation and convergence: simulations of step and frequency responses. , 1992, Optometry and vision science : official publication of the American Academy of Optometry.

[69]  Joseph J. LaViola,et al.  Evaluating user performance in 3D stereo and motion enabled video games , 2012, FDG.

[70]  Alexis Souchet,et al.  Investigating Cyclical Stereoscopy Effects Over Visual Discomfort and Fatigue in Virtual Reality While Learning , 2019, 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[71]  E. I. Sawin.  Taxonomy of Educational Objectives: The Classification of Educational Goals. Handbook 1. Committee of College and University Examiners, Benjamin S. Bloom , 1957 .

[72]  Rustin Webster,et al.  Declarative knowledge acquisition in immersive virtual learning environments , 2016, Interact. Learn. Environ..

[73]  Catherine Plaisant,et al.  Virtual memory palaces: immersion aids recall , 2018, Virtual Reality.

[74]  Hans W. Giessen,et al.  Serious Games Effects: An Overview☆ , 2015 .

[75]  Philippe Fuchs,et al.  Virtual Reality Headsets - A Theoretical and Pragmatic Approach , 2017 .

[76]  Douglas B. Clark,et al.  Digital Games, Design, and Learning , 2016, Review of educational research.

[77]  Joseph R Keebler,et al.  Effects Of 3d Stereoscopy, Visuo-spatial Working Memory, And Perceptions Of Simulation Experience On The Memorization Of Confusable Objects , 2011 .

[78]  Miles E. Hansard,et al.  Causes of discomfort in stereoscopic content: a review , 2017, ArXiv.

[79]  Patrick Gebhard,et al.  Games are Better than Books: In-Situ Comparison of an Interactive Job Interview Game with Conventional Training , 2015, AIED.

[80]  Christoph Rößing,et al.  Human Visual Perception , 2016 .

[81]  Alec M. Bodzin,et al.  Investigating Engagement and Flow with a Placed-Based Immersive Virtual Reality Game , 2020, Journal of Science Education and Technology.

[82]  Neil Jordan,et al.  Mechanism of Action for Obtaining Job Offers With Virtual Reality Job Interview Training. , 2017, Psychiatric services.

[83]  Ananda Bibek Ray,et al.  Smartphone Based Virtual Reality Systems in Classroom Teaching — A Study on the Effects of Learning Outcome , 2016, 2016 IEEE Eighth International Conference on Technology for Education (T4E).

[84]  Kay Stanney,et al.  Virtual Reality Is Sexist: But It Does Not Have to Be , 2020, Frontiers in Robotics and AI.

[85]  D'nardo Colucci,et al.  Perception in HMDs: what is it in head-mounted displays (HMDs) that really make them all so terrible? , 1998, Defense, Security, and Sensing.

[86]  Chetan S. Sankar,et al.  Let Them Play: The Impact of Mechanics and Dynamics of a Serious Game on Student Perceptions of Learning Engagement , 2017, IEEE Transactions on Learning Technologies.

[87]  Pawel Rotter.  Why Did the 3D Revolution Fail?: The Present and Future of Stereoscopy [Commentary] , 2017, IEEE Technol. Soc. Mag..

[88]  Cheryl I. Johnson,et al.  Presence and Usability Do Not Directly Predict Procedural Recall in Virtual Reality Training , 2017, HCI.

[89]  Z. Kapoula,et al.  Induced vergence-accommodation conflict reduces cognitive performance in the Stroop test , 2019, Scientific Reports.

[90]  T Iwasaki,et al.  Effects of a visual task with cognitive demand on dynamic and steady‐state accommodation , 1993, Ophthalmic & physiological optics : the journal of the British College of Ophthalmic Opticians.

[91]  Yan Wang,et al.  Assessment of eye fatigue caused by head-mounted displays using eye-tracking , 2019, Biomedical engineering online.

[92]  Yongtian Wang,et al.  Effects of using HMDs on visual fatigue in virtual environments , 2017, 2017 IEEE Virtual Reality (VR).

[93]  Eric Jamet,et al.  Effects of Stereoscopic Display on Learning and User Experience in an Educational Virtual Environment , 2017, Int. J. Hum. Comput. Interact..

[94]  Philippe Fuchs,et al.  The Challenges and Risks of Democratization of VR‐AR , 2018 .

[95]  M. Mon-Williams,et al.  Binocular vision in a virtual world: visual deficits following the wearing of a head‐mounted display , 1993, Ophthalmic & physiological optics : the journal of the British College of Ophthalmic Opticians.

[96]  Byron J. Pierce,et al.  Perceptual Issues in the Use of Head-Mounted Visual Displays , 2006, Hum. Factors.

[97]  Yongtian Wang,et al.  Subjective and objective evaluation of visual fatigue caused by continuous and discontinuous use of HMDs , 2018, Journal of the Society for Information Display.

[98]  Robert Patterson,et al.  Review Paper: Human factors of stereo displays: An update , 2009 .

[99]  Takeshi Morimoto,et al.  Comparison of visual fatigue caused by head-mounted display for virtual reality and two-dimensional display using objective and subjective evaluation , 2019, Ergonomics.

[100]  S K Rushton,et al.  Developing visual systems and exposure to virtual reality and stereo displays: some concerns and speculations about the demands on accommodation and vergence. , 1999, Applied ergonomics.

[101]  Zulekha Mohamed Elias,et al.  Virtual reality games on accommodation and convergence. , 2019, Applied ergonomics.

[102]  Eduardo Salas,et al.  Game-based Learning: The Impact of Flow State and Videogame Self-efficacy , 2010 .

[103]  Eric D. Ragan,et al.  Higher Levels of Immersion Improve Procedure Memorization Performance , 2009, EGVE/ICAT/EuroVR.

[104]  S. Engeser,et al.  Flow, performance and moderators of challenge-skill balance , 2008 .

[105]  Hee-Sun Lee,et al.  Comparing Short- and Long-Term Learning Effects between Stereoscopic and Two-Dimensional Film at a Planetarium. , 2015 .

[106]  Claudia Schrader,et al.  The influence of virtual presence: Effects on experienced cognitive load and learning outcomes in educational computer games , 2012, Comput. Hum. Behav..

[107]  Mariano Alcañiz Raya,et al.  Learning in serious virtual worlds: Evaluation of learning effectiveness and appeal to students in the E-Junior project , 2010, Comput. Educ..

[108]  Guido Makransky,et al.  Development and validation of the Multimodal Presence Scale for virtual reality environments: A confirmatory factor analysis and item response theory approach , 2017, Comput. Hum. Behav..

[109]  R. Mayer,et al.  Learning Science in Virtual Reality Multimedia Environments: Role of Methods and Media , 2002 .

[110]  Shawn A. Weil,et al.  From Gaming to Training: A Review of Studies on Fidelity, Immersion, Presence, and Buy-in and Their Effects on Transfer in PC-Based Simulations and Games , 2005 .

[111]  Y. Jing,et al.  The Visual Effects Associated with Head-Mounted Displays , 2018, International Journal of Ophthalmology and Clinical Research.

[112]  P A Hancock,et al.  The Effects of Virtual Reality, Augmented Reality, and Mixed Reality as Training Enhancement Methods: A Meta-Analysis , 2020, Hum. Factors.

[113]  David M. Hoffman,et al.  Vergence-accommodation conflicts hinder visual performance and cause visual fatigue. , 2008, Journal of vision.

[114]  Jodi Asbell-Clarke,et al.  Challenging games help students learn: An empirical study on engagement, flow and immersion in game-based learning , 2016, Comput. Hum. Behav..

[115]  A. Munsamy,et al.  The effect of gaming on accommodative and vergence facilities after exposure to virtual reality head-mounted display , 2020, Journal of optometry.

[116]  Yoshio Yamagishi,et al.  Effects of Stereoscopic 3D Contents on the Process of Learning to Build a Handmade PC , 2011 .

[117]  H. Feussner,et al.  Comparative evaluation of HD 2D/3D laparoscopic monitors and benchmarking to a theoretically ideal 3D pseudodisplay: even well-experienced laparoscopists perform better with 3D , 2014, Surgical Endoscopy.