Space-Time Visual Analytics of Eye-Tracking Data for Dynamic Stimuli

We introduce a visual analytics method to analyze eye movement data recorded for dynamic stimuli such as video or animated graphics. The focus lies on the analysis of data of several viewers to identify trends in the general viewing behavior, including time sequences of attentional synchrony and objects with strong attentional focus. By using a space-time cube visualization in combination with clustering, the dynamic stimuli and associated eye gazes can be analyzed in a static 3D representation. Shot-based, spatiotemporal clustering of the data generates potential areas of interest that can be filtered interactively. We also facilitate data drill-down: the gaze points are shown with density-based color mapping and individual scan paths as lines in the space-time cube. The analytical process is supported by multiple coordinated views that allow the user to focus on different aspects of spatial and temporal information in eye gaze data. Common eye-tracking visualization techniques are extended to incorporate the spatiotemporal characteristics of the data. For example, heat maps are extended to motion-compensated heat maps and trajectories of scan paths are included in the space-time visualization. Our visual analytics approach is assessed in a qualitative users study with expert users, which showed the usefulness of the approach and uncovered that the experts applied different analysis strategies supported by the system.

[1]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[2]  Min Chen,et al.  Action-Based Multifield Video Visualization , 2008, IEEE Transactions on Visualization and Computer Graphics.

[3]  L. Itti,et al.  Visual causes versus correlates of attentional selection in dynamic scenes , 2006, Vision Research.

[4]  Linden J. Ball,et al.  Eye tracking in HCI and usability research. , 2006 .

[5]  Anoop Gupta,et al.  Browsing digital video , 2000, CHI.

[6]  Bruce H. McCormick,et al.  Gaze-contingent video resolution degradation , 1998, Electronic Imaging.

[7]  Gordon Erlebacher,et al.  Overview of Flow Visualization , 2005, The Visualization Handbook.

[8]  D. S. Wooding,et al.  Fixation maps: quantifying eye-movement traces , 2002, ETRA.

[9]  Michael Burch,et al.  Visual Analytics Methodology for Eye Movement Studies , 2012, IEEE Transactions on Visualization and Computer Graphics.

[10]  Melanie Tory,et al.  Comparing ExoVis, Orientation Icon, and In-Place 3D Visualization Techniques , 2003, Graphics Interface.

[11]  Pilar Orero,et al.  Aggregate gaze visualization with real-time heatmaps , 2012, ETRA.

[12]  Thomas Ertl,et al.  Parallel scan-path visualization , 2012, ETRA.

[13]  John M. Henderson,et al.  Clustering of Gaze During Dynamic Scene Viewing is Predicted by Motion , 2011, Cognitive Computation.

[14]  Heidrun Schumann,et al.  Visualization of Time-Oriented Data , 2011, Human-Computer Interaction Series.

[15]  J.C. Roberts,et al.  State of the Art: Coordinated & Multiple Views in Exploratory Visualization , 2007, Fifth International Conference on Coordinated and Multiple Views in Exploratory Visualization (CMV 2007).

[16]  Andrew T Duchowski,et al.  A breadth-first survey of eye-tracking applications , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[17]  Eli Peli,et al.  Where people look when watching movies: Do all viewers look at the same place? , 2007, Comput. Biol. Medicine.

[18]  Melanie Tory,et al.  eSeeTrack—Visualizing Sequential Fixation Patterns , 2010, IEEE Transactions on Visualization and Computer Graphics.

[19]  Daniel A. Keim,et al.  Visual Analytics: Scope and Challenges , 2008, Visual Data Mining.

[20]  M. Stella Atkins,et al.  Measuring gaze overlap on videos between multiple observers , 2012, ETRA.

[21]  Kenneth Holmqvist,et al.  Eye tracking: a comprehensive guide to methods and measures , 2011 .

[22]  Eric Bruno,et al.  Video shot detection based on linear prediction of motion , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[23]  Douglas DeCarlo,et al.  Robust clustering of eye movement recordings for quantification of visual interest , 2004, ETRA.

[24]  L. Itti,et al.  Quantifying center bias of observers in free viewing of dynamic natural scenes. , 2009, Journal of vision.

[25]  Margit Pohl,et al.  Visualisation and Analysis of Multiuser Gaze Data: Eye Tracking Usability Studies in the Special Context of E-learning , 2012, 2012 IEEE 12th International Conference on Advanced Learning Technologies.

[26]  John M. Henderson,et al.  Attentional synchrony in static and dynamic scenes , 2010 .

[27]  TsangHoi Ying,et al.  eSeeTrack—Visualizing Sequential Fixation Patterns , 2010 .

[28]  Michael Burch,et al.  Evaluation of Traditional, Orthogonal, and Radial Tree Diagrams by an Eye Tracking Study , 2011, IEEE Transactions on Visualization and Computer Graphics.

[29]  Paul Over,et al.  Video shot boundary detection: Seven years of TRECVid activity , 2010, Comput. Vis. Image Underst..

[30]  Cynthia A. Brewer,et al.  ColorBrewer.org: An Online Tool for Selecting Colour Schemes for Maps , 2003 .

[31]  Agnieszka Bojko,et al.  Informative or Misleading? Heatmaps Deconstructed , 2009, HCI.

[32]  Min Chen,et al.  Visual Signatures in Video Visualization , 2006, IEEE Transactions on Visualization and Computer Graphics.

[33]  Maarten van Someren,et al.  The Think Aloud Method: A Practical Guide to Modelling Cognitive Processes , 1994 .

[34]  B. Tatler,et al.  Yarbus, eye movements, and vision , 2010, i-Perception.

[35]  Marcus B. Perry,et al.  The Exponentially Weighted Moving Average , 2010 .

[36]  Gunnar Farnebäck,et al.  Two-Frame Motion Estimation Based on Polynomial Expansion , 2003, SCIA.

[37]  Nilesh V. Patel,et al.  Video shot detection and characterization for video databases , 1997, Pattern Recognit..

[38]  Ben Shneiderman,et al.  The eyes have it: a task by data type taxonomy for information visualizations , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.

[39]  Pieter J. Blignaut,et al.  Visual span and other parameters for the generation of heatmaps , 2010, ETRA.

[40]  Richard Stevens,et al.  Are you seeing what I'm seeing? An eye-tracking evaluation of dynamic scenes , 2009, Digit. Creativity.

[41]  Nobuyuki Hiruma,et al.  Determining comprehension and quality of TV programs using eye-gaze tracking , 2008, Pattern Recognit..

[42]  Gennady L. Andrienko,et al.  Interactive analysis of event data using space-time cube , 2004, Proceedings. Eighth International Conference on Information Visualisation, 2004. IV 2004..

[43]  Xia Li,et al.  Visual Exploration of Eye Movement Data Using the Space-Time-Cube , 2010, GIScience.

[44]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[45]  Joseph H. Goldberg,et al.  Identifying fixations and saccades in eye-tracking protocols , 2000, ETRA.