Eye tracking evaluation of visual analytics

The application of eye tracking for the evaluation of humans’ viewing behavior is a common approach in psychological research. So far, the use of this technique for the evaluation of visual analytics and visualization is less prominent. We investigate recent scientific publications from the main visualization and visual analytics conferences and journals, as well as related research fields that include an evaluation by eye tracking. Furthermore, we provide an overview of evaluation goals that can be achieved by eye tracking and state-of-the-art analysis techniques for eye tracking data. Ideally, visual analytics leads to a mixed-initiative cognitive system where the mechanism of distribution is the interaction of the user with the visualization environment. Therefore, we also include a discussion of cognitive approaches and models to include the user in the evaluation process. Based on our review of the current use of eye tracking evaluation in our field and the cognitive theory, we propose directions for future research on evaluation methodology, leading to the grand challenge of developing an evaluation approach to the mixed-initiative cognitive system of visual analytics.

[1]  I. Biederman Perceiving Real-World Scenes , 1972, Science.

[2]  Dominique L. Scapin,et al.  User-Centered Evaluation of Information Visualization Techniques: Making the HCI-InfoVis Connection Explicit , 2014, Handbook of Human Centric Visualization.

[3]  M. Sheelagh T. Carpendale,et al.  Evaluating Information Visualizations , 2008, Information Visualization.

[4]  Francis K. H. Quek Eyes in the interface , 1995, Image Vis. Comput..

[5]  Kristin A. Cook,et al.  Illuminating the Path: The Research and Development Agenda for Visual Analytics , 2005 .

[6]  Lars Linsen,et al.  EyeC: Coordinated Views for Interactive Visual Exploration of Eye-Tracking Data , 2013, 2013 17th International Conference on Information Visualisation.

[7]  A. L. Yarbus,et al.  Eye Movements and Vision , 1967, Springer US.

[8]  James T Enns,et al.  Multiple-object tracking is based on scene, not retinal, coordinates. , 2005, Journal of experimental psychology. Human perception and performance.

[9]  Antonio Torralba,et al.  Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. , 2006, Psychological review.

[10]  Michael Burch,et al.  Visual task solution strategies in tree diagrams , 2013, 2013 IEEE Pacific Visualization Symposium (PacificVis).

[11]  Tobias Isenberg,et al.  A Systematic Review on the Practice of Evaluating Visualization , 2013, IEEE Transactions on Visualization and Computer Graphics.

[12]  Heidrun Schumann,et al.  A Design Space of Visualization Tasks , 2013, IEEE Transactions on Visualization and Computer Graphics.

[13]  William Ribarsky,et al.  Science of Analytical Reasoning , 2009, Inf. Vis..

[14]  C. Trevarthen,et al.  Two mechanisms of vision in primates , 1968, Psychologische Forschung.

[15]  Thomas Ertl,et al.  Towards analyzing eye tracking data for evaluating interactive visualization systems , 2014, BELIV.

[16]  Michael Burch,et al.  Evaluation of Traditional, Orthogonal, and Radial Tree Diagrams by an Eye Tracking Study , 2011, IEEE Transactions on Visualization and Computer Graphics.

[17]  Anatole Lécuyer,et al.  Design and Application of Real-Time Visual Attention Model for the Exploration of 3D Virtual Environments , 2012, IEEE Transactions on Visualization and Computer Graphics.

[18]  Rajesh P. N. Rao,et al.  PSYCHOLOGICAL SCIENCE Research Article EYE MOVEMENTS REVEAL THE SPATIOTEMPORAL DYNAMICS OE VISUAL SEARCH , 2022 .

[19]  Benjamin Naumann The Architecture Of Cognition , 2016 .

[20]  Weidong Huang,et al.  Using eye tracking to investigate graph layout effects , 2007, 2007 6th International Asia-Pacific Symposium on Visualization.

[21]  Jarke J. van Wijk,et al.  Evaluation: A Challenge for Visual Analytics , 2013, Computer.

[22]  Sung-Hee Kim,et al.  Does an Eye Tracker Tell the Truth about Visualizations?: Findings while Investigating Visualizations for Decision Making , 2012, IEEE Transactions on Visualization and Computer Graphics.

[23]  Chris North,et al.  Toward measuring visualization insight , 2006, IEEE Computer Graphics and Applications.

[24]  Richard A. Bolt Eyes at the interface , 1982, CHI '82.

[25]  Amitabh Varshney,et al.  Persuading Visual Attention through Geometry , 2008, IEEE Transactions on Visualization and Computer Graphics.

[26]  Harri Siirtola,et al.  Visual Perception of Parallel Coordinate Visualizations , 2009, 2009 13th International Conference Information Visualisation.

[27]  Linden J. Ball,et al.  Eye Tracking in Human-Computer Interaction and Usability Research : Current Status and Future Prospects , 2004 .

[28]  Michael Burch,et al.  A visual approach for scan path comparison , 2014, ETRA.

[29]  Ron Sun,et al.  From implicit skills to explicit knowledge: a bottom-up model of skill learning , 2001, Cogn. Sci..

[30]  N. Charness,et al.  Visual Span in Expert Chess Players: Evidence From Eye Movements , 2001, Psychological science.

[31]  Ryen W. White,et al.  No clicks, no problem: using cursor movements to understand and improve search , 2011, CHI.

[32]  Jinwook Seo,et al.  GazeVis: Interactive 3D Gaze Visualization for Contiguous Cross-Sectional Medical Images , 2014, IEEE Transactions on Visualization and Computer Graphics.

[33]  Michael Burch,et al.  State-of-the-Art of Visualization for Eye Tracking Data , 2014, EuroVis.

[34]  Anthony J. Hornof,et al.  Cognitive strategies and eye movements for searching hierarchical computer displays , 2003, CHI '03.

[35]  Daniel Weiskopf,et al.  AOI Rivers for Visualizing Dynamic Eye Gaze Frequencies , 2013, Comput. Graph. Forum.

[36]  Marianne A. DeAngelus,et al.  Top-down control of eye movements: Yarbus revisited , 2009 .

[37]  Catherine Plaisant,et al.  The challenge of information visualization evaluation , 2004, AVI.

[38]  Thomas Ertl,et al.  Visual analysis of perceptual and cognitive processes , 2014, 2014 International Conference on Information Visualization Theory and Applications (IVAPP).

[39]  Andrew L. Kun,et al.  Estimating cognitive load using remote eye tracking in a driving simulator , 2010, ETRA.

[40]  M. Sheelagh T. Carpendale,et al.  Empirical Studies in Information Visualization: Seven Scenarios , 2012, IEEE Transactions on Visualization and Computer Graphics.

[41]  Andrew T. Duchowski,et al.  Group-wise similarity and classification of aggregate scanpaths , 2010, ETRA '10.

[42]  Gitta Domik,et al.  Building Bridges: HCI, Visualization, and Non-formal Modeling , 2014, Lecture Notes in Computer Science.

[43]  Julie C. Sedivy,et al.  Subject Terms: Linguistics Language Eyes & eyesight Cognition & reasoning , 1995 .

[44]  Florian Heimerl,et al.  ISeeCube: visual analysis of gaze data for video , 2014 .

[45]  Esma Aïmeur,et al.  Activity recognition using eye-gaze movements and traditional interactions , 2011, Interact. Comput..

[46]  Anthony J. Hornof,et al.  A minimal model for predicting visual search in human-computer interaction , 2007, CHI.

[47]  Dario D. Salvucci Inferring intent in eye-based interfaces: tracing eye movements with process models , 1999, CHI '99.

[48]  A. L. I︠A︡rbus Eye Movements and Vision , 1967 .

[49]  Pat Hanrahan,et al.  Measuring the task-evoked pupillary response with a remote eye tracker , 2008, ETRA.

[50]  Melanie Tory User Studies in Visualization: A Reflection on Methods , 2014, Handbook of Human Centric Visualization.

[51]  David E. Kieras,et al.  An Overview of the EPIC Architecture for Cognition and Performance With Application to Human-Computer Interaction , 1997, Hum. Comput. Interact..

[52]  Andrew T Duchowski,et al.  A breadth-first survey of eye-tracking applications , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[53]  Daniel Weiskopf,et al.  Evaluation of Attention‐Guiding Video Visualization , 2013, Comput. Graph. Forum.

[54]  John R Anderson,et al.  An integrated theory of the mind. , 2004, Psychological review.

[55]  Robert Spence,et al.  The Broker , 2009, HCIV.

[56]  Joseph H. Goldberg,et al.  Eye tracking for visualization evaluation: Reading values on linear versus radial graphs , 2011, Inf. Vis..

[57]  Erik W. Anderson Evaluating visualization using cognitive measures , 2012, BELIV '12.

[58]  Niklas Elmqvist,et al.  Patterns for visualization evaluation , 2015, Inf. Vis..

[59]  Veikko Surakka,et al.  Pupillary responses to emotionally provocative stimuli , 2000, ETRA.

[60]  Michael Burch,et al.  Evaluating visual analytics with eye tracking , 2014, BELIV.

[61]  Susan Leigh Star,et al.  The Structure of Ill-Structured Solutions: Boundary Objects and Heterogeneous Distributed Problem Solving , 1989, Distributed Artificial Intelligence.

[62]  John R. Anderson,et al.  Eye tracking the visual search of click-down menus , 1999, CHI '99.

[63]  Weidong Huang,et al.  How People Read Graphs , 2005, APVIS.

[64]  K. A. Ericsson,et al.  Protocol Analysis: Verbal Reports as Data , 1984 .

[65]  Melanie Tory,et al.  Comparing Parameter Manipulation with Mouse, Pen, and Slider User Interfaces , 2009, Comput. Graph. Forum.

[66]  Alan Kennedy,et al.  Book Review: Eye Tracking: A Comprehensive Guide to Methods and Measures , 2016, Quarterly journal of experimental psychology.

[67]  Joseph H. Goldberg,et al.  Comparing information graphics: a critical look at eye tracking , 2010, BELIV '10.

[68]  Brian D. Ehret,et al.  Learning where to look: location learning in graphical user interfaces , 2002, CHI.

[69]  Nilanjan Sarkar,et al.  Understanding How Adolescents with Autism Respond to Facial Expressions in Virtual Reality Environments , 2013, IEEE Transactions on Visualization and Computer Graphics.

[70]  Michael Burch,et al.  Comparative Eye Tracking Study on Node-Link Visualizations of Trajectories , 2014, IEEE Transactions on Visualization and Computer Graphics.

[71]  Sebastian Koch,et al.  Visual Analysis and Dissemination of Scientific Literature Collections with SurVis , 2016, IEEE Transactions on Visualization and Computer Graphics.

[72]  Marcus A. Magnor,et al.  Temporal Video Filtering and Exposure Control for Perceptual Motion Blur , 2015, IEEE Transactions on Visualization and Computer Graphics.

[73]  Brian P. Bailey,et al.  Towards an index of opportunity: understanding changes in mental workload during task execution , 2004, CHI.

[74]  Yifan Hu,et al.  How to Display Group Information on Node-Link Diagrams: An Evaluation , 2014, IEEE Transactions on Visualization and Computer Graphics.

[75]  Linden J. Ball,et al.  Eye tracking in HCI and usability research. , 2006 .

[76]  Leo Lentz,et al.  Retrospective think-aloud method: using eye movements as an extra cue for participants' verbalizations , 2011, CHI.

[77]  Michael Burch,et al.  Visual Analytics Methodology for Eye Movement Studies , 2012, IEEE Transactions on Visualization and Computer Graphics.

[78]  Brian D. Fisher,et al.  Pointing and Visual Feedback for Spatial Interaction in Large-Screen Display Environments , 2003, Smart Graphics.

[79]  Allen Newell,et al.  The psychology of human-computer interaction , 1983 .

[80]  Daniel Weiskopf,et al.  Space-Time Visual Analytics of Eye-Tracking Data for Dynamic Stimuli , 2013, IEEE Transactions on Visualization and Computer Graphics.

[81]  Laurent Itti,et al.  Computational mechanisms for gaze direction in interactive visual environments , 2006, ETRA.

[82]  Anthony C. Robinson,et al.  Comparing Color and Leader Line Highlighting Strategies in Coordinated View Geovisualizations , 2015, IEEE Transactions on Visualization and Computer Graphics.

[83]  Zhiwei Guan,et al.  The validity of the stimulated retrospective think-aloud method as measured by eye tracking , 2006, CHI.

[84]  Edward Cutrell,et al.  An eye tracking study of the effect of target rank on web search , 2007, CHI.

[85]  Guang-Zhong Yang,et al.  Eyegaze analysis of displays with combined 2D and 3D views , 2005, VIS 05. IEEE Visualization, 2005..

[86]  Alistair G. Sutcliffe,et al.  Designing effective multimedia presentations , 1997, CHI.

[87]  Colin Ware,et al.  Visual Queries: The Foundation of Visual Thinking , 2005, Knowledge and Information Visualization.

[88]  N. Mackworth,et al.  Cognitive determinants of fixation location during picture viewing. , 1978, Journal of experimental psychology. Human perception and performance.

[89]  Brian D. Fisher,et al.  Visual Analytics as a Translational Cognitive Science , 2011, Top. Cogn. Sci..

[90]  Melanie Tory,et al.  eSeeTrack—Visualizing Sequential Fixation Patterns , 2010, IEEE Transactions on Visualization and Computer Graphics.

[91]  I. Biederman,et al.  Viewpoint-dependent mechanisms in visual object recognition: Reply to Tarr and Bülthoff (1995). , 1995 .

[92]  Zenon W. Pylyshyn,et al.  Situating vision in the world , 2000, Trends in Cognitive Sciences.