Predicting cybersickness based on user's gaze behaviors in HMD-based virtual reality

Cybersickness refers to a group of uncomfortable symptoms experienced in virtual reality (VR). Among several theories of cybersickness, the subjective vertical mismatch (SVM) theory focuses on an individual’s internal model, which is created and updated through past experiences. Although previous studies have attempted to provide experimental evidence for the theory, most approaches are limited to subjective measures or body sway. In this study, we aimed to demonstrate the SVM theory on the basis of the participant’s eye movements and investigate whether the subjective level of cybersickness can be predicted using eye-related measures. 26 participants experienced roller coaster VR while wearing a head-mounted display with eye tracking. We designed four experimental conditions by changing the orientation of the VR scene (upright vs. inverted) or the controllability of the participant’s body (unrestrained vs. restrained body). The results indicated that participants reported more severe cybersickness when experiencing the upright VR content without controllability. Moreover, distinctive eye movements (e.g. fixation duration and distance between the eye gaze and the object position sequence) were observed according to the experimental conditions. On the basis of these results, we developed a regression model using eye-movement features and found that our model can explain 34.8% of the total variance of cybersickness, indicating a substantial improvement compared to the previous work (4.2%). This study provides empirical data for the SVM theory using both subjective and eye-related measures. In particular, the results suggest that participants’ eye movements can serve as a significant index for predicting cybersickness when considering natural gaze behaviors during a VR experience.

[1]  Alessandro Ceruti,et al.  Maintenance in aeronautics in an Industry 4.0 context: The role of Augmented Reality and Additive Manufacturing , 2019, J. Comput. Des. Eng..

[2]  Nobuyoshi Yabuki,et al.  An indoor thermal environment design system for renovation using augmented reality , 2018, J. Comput. Des. Eng..

[3]  Steven K. Feiner,et al.  Combating VR sickness through subtle dynamic field-of-view modification , 2016, 2016 IEEE Symposium on 3D User Interfaces (3DUI).

[4]  Gordon Wetzstein,et al.  Towards a Machine-Learning Approach for Sickness Prediction in 360° Stereoscopic Videos , 2018, IEEE Transactions on Visualization and Computer Graphics.

[5]  T A Furness,et al.  The use of an independent visual background to reduce simulator side-effects. , 1999, Aviation, space, and environmental medicine.

[6]  Stephen Palmisano,et al.  Effects of postural stability, active control, exposure duration and repeated exposures on HMD induced cybersickness , 2019, Displays.

[7]  J. Golding,et al.  Cognitive cues and visually induced motion sickness. , 2012, Aviation, space, and environmental medicine.

[8]  Robert W. Lindeman,et al.  Investigating a Physical Dial as a Measurement Tool for Cybersickness in Virtual Reality , 2019, VRST.

[9]  Heiko Hecht,et al.  Axis rotation and visually induced motion sickness: the role of combined roll, pitch, and yaw motion. , 2011, Aviation, space, and environmental medicine.

[10]  Yong Man Ro,et al.  Deep Objective Assessment Model Based on Spatio-Temporal Perception of 360-Degree Video for VR Sickness Prediction , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[11]  Young Youn Kim,et al.  Characteristic changes in the physiological components of cybersickness. , 2005, Psychophysiology.

[12]  Séamas Weech,et al.  Estimating the sensorimotor components of cybersickness , 2018, Journal of neurophysiology.

[13]  Robert W. Lindeman,et al.  Exploring natural eye-gaze-based interaction for immersive virtual reality , 2017, 2017 IEEE Symposium on 3D User Interfaces (3DUI).

[14]  Antje Nuthmann,et al.  Asymmetrical control of fixation durations in scene viewing , 2014, Vision Research.

[15]  Cyriel Diels,et al.  Self-driving carsickness. , 2016, Applied ergonomics.

[16]  Jelte E. Bos,et al.  Motion in images is essential to cause motion sickness symptoms, but not to increase postural sway , 2015, Displays.

[17]  Kang-Hee Lee,et al.  TapSix: A Palm-Worn Glove with a Low-Cost Camera Sensor that Turns a Tactile Surface into a Six-Key Chorded Keyboard by Detection Finger Taps , 2020, Int. J. Hum. Comput. Interact..

[18]  Paulus Insap Santosa,et al.  Physical discomfort and eye movements during arbitrary and optical flow-like motions in stereo 3D contents , 2019, Virtual Reality.

[19]  Ronen Hershman,et al.  A novel blink detection method based on pupillometry noise , 2018, Behavior Research Methods.

[20]  F. Bonato,et al.  Expanding and contracting optical flow patterns and simulator sickness. , 2007, Aviation, space, and environmental medicine.

[21]  Meredith Ringel Morris,et al.  Toward Everyday Gaze Input: Accuracy and Precision of Eye Tracking and Implications for Design , 2017, CHI.

[22]  Andreas Paepcke,et al.  Improving the accuracy of gaze input for interaction , 2008, ETRA.

[23]  J. Bos,et al.  The Effect of Optokinetic Stimulation on Perceptual and Postural Symptoms in Visual Vestibular Mismatch Patients , 2016, PloS one.

[24]  Steven G. Luke,et al.  The Influence of Content Meaningfulness on Eye Movements across Tasks: Evidence from Scene Viewing and Reading , 2016, Front. Psychol..

[25]  Manuel Contero,et al.  I walk, therefore I am: a multidimensional study on the influence of the locomotion method upon presence in virtual reality , 2020, J. Comput. Des. Eng..

[26]  Shunnan Yang,et al.  Effects of vergence and accommodative responses on viewer's comfort in viewing 3D stimuli , 2011, Electronic Imaging.

[27]  Jelte E. Bos,et al.  A theory on visually induced motion sickness , 2008, Displays.

[28]  Robert S. Kennedy,et al.  Simulator Sickness Questionnaire: An enhanced method for quantifying simulator sickness. , 1993 .

[29]  Yong Man Ro,et al.  VRSA Net: VR Sickness Assessment Considering Exceptional Motion for 360° VR Video , 2019, IEEE Transactions on Image Processing.

[30]  T. Stoffregen,et al.  Postural instability precedes motion sickness , 1998, Brain Research Bulletin.

[31]  Li-Yi Wei,et al.  Reducing simulator sickness with perceptual camera control , 2019, ACM Trans. Graph..

[32]  Philippe Pasquier,et al.  Automatic Prediction of Cybersickness for Virtual Reality Games , 2018, 2018 IEEE Games, Entertainment, Media Conference (GEM).

[33]  A. Nuthmann Fixation durations in scene viewing: Modeling the effects of local image features, oculomotor parameters, and task , 2016, Psychonomic bulletin & review.

[34]  K. Ukai,et al.  Visually induced motion sickness with radial displays: effects of gaze angle and fixation. , 2007, Aviation, space, and environmental medicine.

[35]  Sarah Sharples,et al.  Virtual reality induced symptoms and effects (VRISE): Comparison of head mounted display (HMD), desktop and projection display systems , 2008, Displays.

[36]  Dinesh Manocha,et al.  DGaze: CNN-Based Gaze Prediction in Dynamic Scenes , 2020, IEEE Transactions on Visualization and Computer Graphics.

[37]  Richard H. Y. So,et al.  Visually induced motion sickness: Effects of translational visual motion along different axes , 2011 .

[38]  Mark S. Dennison,et al.  Use of physiological signals to predict cybersickness , 2016, Displays.

[39]  Claudio Robazza,et al.  Action Monitoring Through External or Internal Focus of Attention Does Not Impair Endurance Performance , 2019, Front. Psychol..

[40]  M. Johnson,et al.  Parsing eye-tracking data of variable quality to provide accurate fixation duration estimates in infants and adults , 2012, Behavior research methods.

[41]  Katherine Demuth,et al.  Effects of Type of Agreement Violation and Utterance Position on the Auditory Processing of Subject-Verb Agreement: An ERP Study , 2016, Front. Psychol..

[42]  Taehyun Rhee,et al.  Reducing Visual Discomfort with HMDs Using Dynamic Depth of Field , 2015, IEEE Computer Graphics and Applications.

[43]  Heiko Hecht,et al.  Vection is the main contributor to motion sickness induced by visual yaw rotation: Implications for conflict and eye movement theories , 2017, PloS one.

[44]  Stephen A. Palmisano,et al.  Vection Change Exacerbates Simulator Sickness in Virtual Environments , 2008, PRESENCE: Teleoperators and Virtual Environments.

[45]  Oleg V. Komogortsev,et al.  An implementation of eye movement-driven biometrics in virtual reality , 2018, ETRA.

[46]  Satoru Fujisawa,et al.  Analysis of driver's head tilt using a mathematical model of motion sickness , 2016, 1711.00747.

[47]  Taylor R. Hayes,et al.  Eye Movements in Real-World Scene Photographs: General Characteristics and Effects of Viewing Task , 2020, Frontiers in Psychology.

[48]  Ronald R. Mourant,et al.  Comparison of Simulator Sickness Using Static and Dynamic Walking Simulators , 2001 .

[49]  J. Bos,et al.  Framing visual roll-motion affects postural sway and the subjective visual vertical , 2016, Attention, perception & psychophysics.

[50]  Wei Hu,et al.  Navigation modes, operation methods, observation scales and background options in UI design for high learning performance in VR-based architectural applications , 2018, J. Comput. Des. Eng..

[51]  Eunhee Chang,et al.  Virtual Reality Sickness: A Review of Causes and Measurements , 2020, Int. J. Hum. Comput. Interact..