Visualization and Analysis of Head Movement and Gaze Data for Immersive Video in Head-mounted Displays

In contrast to traditional video, immersive video allows viewers to interactively control their field of view in a 360◦ panoramic scene. However, established methods for the comparative evaluation of gaze data for video require that all participants observe the same viewing area. We therefore propose new specialized visualizations and a novel visual analytics workflow for the combined analysis of head movement and gaze data. A View Similarity visualization highlights viewing areas branching and joining over time, while three additional visualizations provide global and spatial context. These new visualizations, along with established gaze evaluation techniques, allow analysts to investigate the storytelling of immersive video. We demonstrate the usefulness of our approach using head movement and gaze data recorded for both amateur panoramic videos, as well as professionally composited immersive videos. Index Terms —Visual analytics, eye tracking, immersive video

[1]  N H MACKWORTH,et al.  Eye fixations recorded on changing visual scenes by the television eye-marker. , 1958, Journal of the Optical Society of America.

[2]  L. Stark,et al.  Scanpaths in Eye Movements during Pattern Perception , 1971, Science.

[3]  Ken Shoemake,et al.  ARCBALL: a user interface for specifying three-dimensional orientation using a mouse , 1992 .

[4]  John Paulin Hansen,et al.  COGNITIVE MODELLING OF A SHIP NAVIGATOR BASED ON PROTOCOL AND EYE-MOVEMENT ANALYSIS , 1998 .

[5]  Kenji Itoh,et al.  Eye-Movement Analysis of Track Monitoring Patterns of Night Train Operators: Effects of Geographic Knowledge and Fatigue , 2000 .

[6]  Trevor F. Cox,et al.  Multidimensional Scaling, Second Edition , 2000 .

[7]  Anand K. Gramopadhye,et al.  3D eye movement analysis for VR visual inspection training , 2002, ETRA.

[8]  C. Trepagnier,et al.  Gaze data visualization tools: opportunities and challenges , 2004, Proceedings. Eighth International Conference on Information Visualisation, 2004. IV 2004..

[9]  Guang-Zhong Yang,et al.  Eyegaze analysis of displays with combined 2D and 3D views , 2005, VIS 05. IEEE Visualization, 2005..

[10]  John M. Henderson,et al.  Attentional synchrony in static and dynamic scenes , 2010 .

[11]  Marcus Nyström,et al.  Effect of compressed offline foveated video on viewing behavior and subjective quality , 2010, TOMCCAP.

[12]  Raimund Dachselt,et al.  Advanced gaze visualizations for three-dimensional virtual environments , 2010, ETRA.

[13]  G Schneider,et al.  Visual attention of anaesthetists during simulated critical incidents. , 2011, British journal of anaesthesia.

[14]  Thies Pfeiffer.  Measuring and visualizing attention in space with 3D attention volumes , 2012, ETRA '12.

[15]  Pilar Orero,et al.  Aggregate gaze visualization with real-time heatmaps , 2012, ETRA.

[16]  Nadir Weibel,et al.  Let's look at the cockpit: exploring mobile eye-tracking for observational research on the flight deck , 2012, ETRA.

[17]  Daniel Weiskopf,et al.  Space-Time Visual Analytics of Eye-Tracking Data for Dynamic Stimuli , 2013, IEEE Transactions on Visualization and Computer Graphics.

[18]  Daniel Weiskopf,et al.  AOI Rivers for Visualizing Dynamic Eye Gaze Frequencies , 2013, Comput. Graph. Forum.

[19]  Marcus A. Magnor,et al.  A Nonobscuring Eye Tracking Solution for Wide Field-of-View Head-mounted Displays , 2014, Eurographics.

[20]  Michael Burch,et al.  State-of-the-Art of Visualization for Eye Tracking Data , 2014, EuroVis.

[21]  Markus H. Gross,et al.  Panoramic Video from Unstructured Camera Arrays , 2015, Comput. Graph. Forum.

[22]  Daniel Weiskopf,et al.  AOI transition trees , 2015, Graphics Interface.

[23]  Jong-Seok Lee,et al.  Temporal resolution vs. visual saliency in videos: Analysis of gaze patterns and evaluation of saliency models , 2015, Signal Process. Image Commun..