Gaze Analysis and Prediction in Virtual Reality

In virtual reality (VR) systems, users’ gaze information has gained importance in recent years. It can be applied to many aspects, including VR content design, eye-movement based interaction, gaze-contingent rendering, etc. In this context, it becomes increasingly important to understand users’ gaze behaviors in virtual reality and to predict users’ gaze positions. This paper presents research in gaze behavior analysis and gaze position prediction in virtual reality. Specifically, this paper focuses on static virtual scenes and dynamic virtual scenes under free-viewing conditions. Users’ gaze data in virtual scenes are collected and statistical analysis is performed on the recorded data. The analysis reveals that users’ gaze positions are correlated with their head rotation velocities and the salient regions of the content. In dynamic scenes, users’ gaze positions also have strong correlations with the positions of dynamic objects. A data-driven eye-head coordination model is proposed for real-time gaze prediction in static scenes and a CNN-based model is derived for predicting gaze positions in dynamic scenes.

[1]  A. L. Yarbus,et al.  Eye Movements and Vision , 1967, Springer US.

[2]  George Drettakis,et al.  Gaze prediction using machine learning for dynamic stereo manipulation in games , 2016, 2016 IEEE Virtual Reality (VR).

[3]  Dinesh Manocha,et al.  DGaze: CNN-Based Gaze Prediction in Dynamic Scenes , 2020, IEEE Transactions on Visualization and Computer Graphics.

[4]  A. L. I︠A︡rbus.  Eye Movements and Vision , 1967 .

[5]  Peter König,et al.  Human eye-head co-ordination in natural exploration , 2007, Network.

[6]  Dinesh Manocha,et al.  SGaze: A Data-Driven Eye-Head Coordination Model for Realtime Gaze Prediction , 2019, IEEE Transactions on Visualization and Computer Graphics.

[7]  Gordon Wetzstein,et al.  Saliency in VR: How Do People Explore Virtual Environments? , 2016, IEEE Transactions on Visualization and Computer Graphics.

[8]  Joohwan Kim,et al.  Towards foveated rendering for gaze-tracked virtual reality , 2016, ACM Trans. Graph..

[9]  Rita Cucchiara,et al.  Predicting Human Eye Fixations via an LSTM-Based Saliency Attentive Model , 2016, IEEE Transactions on Image Processing.

[10]  D. Simons,et al.  Moving and looming stimuli capture attention , 2003, Perception & psychophysics.

[11]  Shenghua Gao,et al.  Gaze Prediction in Dynamic 360° Immersive Videos , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[12]  C. Koch,et al.  Models of bottom-up and top-down visual attention , 2000 .