Eye tracking support for visual analytics systems: foundations, current applications, and research challenges

Visual analytics (VA) research provides helpful solutions for interactive visual data analysis when exploring large and complex datasets. Due to recent advances in eye tracking technology, promising opportunities arise to extend these traditional VA approaches. Therefore, we discuss foundations for eye tracking support in VA systems. We first review and discuss the structure and range of typical VA systems. Based on a widely used VA model, we present five comprehensive examples that cover a wide range of usage scenarios. Then, we demonstrate that the VA model can be used to systematically explore how concrete VA systems could be extended with eye tracking, to create supportive and adaptive analytics systems. This allows us to identify general research and application opportunities, and classify them into research themes. In a call for action, we map the road for future research to broaden the use of eye tracking and advance visual analytics.

[1]  Daniel A. Keim,et al.  Transformations of Movement Data , 2013 .

[2]  Gennady L. Andrienko,et al.  Clustering Trajectories by Relevant Parts for Air Traffic Analysis , 2018, IEEE Transactions on Visualization and Computer Graphics.

[3]  Cristina Conati,et al.  User-adaptive information visualization: using eye gaze data to infer visualization tasks and user cognitive abilities , 2013, IUI '13.

[4]  Radu Jianu,et al.  A Data Model and Task Space for Data of Interest (DOI) Eye-Tracking Analyses , 2018, IEEE Transactions on Visualization and Computer Graphics.

[5]  Gennady L. Andrienko,et al.  Analysis of Flight Variability: a Systematic Approach , 2019, IEEE Transactions on Visualization and Computer Graphics.

[6]  Silvia Miksch,et al.  The Role of Explicit Knowledge: A Conceptual Model of Knowledge-Assisted Visual Analytics , 2017, 2017 IEEE Conference on Visual Analytics Science and Technology (VAST).

[7]  Michael Burch,et al.  Evaluating visual analytics with eye tracking , 2014, BELIV.

[8]  Minho Lee,et al.  Evolutionary programming based recommendation system for online shopping , 2013, 2013 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference.

[9]  O. Hikosaka,et al.  What do eye movements tell us about patients with neurological disorders? — An introduction to saccade recording in the clinical setting — , 2017, Proceedings of the Japan Academy. Series B, Physical and biological sciences.

[10]  Qian Zhao,et al.  Gaze Prediction for Recommender Systems , 2016, RecSys.

[11]  Sören Preibusch,et al.  Privacy considerations for a pervasive eye tracking world , 2014, UbiComp Adjunct.

[12]  John T. Stasko,et al.  The Science of Interaction , 2009, Inf. Vis..

[13]  Daniel A. Keim,et al.  Knowledge Generation Model for Visual Analytics , 2014, IEEE Transactions on Visualization and Computer Graphics.

[14]  Jason Dykes,et al.  Visual analysis of pressure in football , 2017, Data Mining and Knowledge Discovery.

[15]  Eugene Agichtein,et al.  Detecting cognitive impairment by eye movement analysis using automatic classification algorithms , 2011, Journal of Neuroscience Methods.

[16]  Daniel A. Keim,et al.  Visual Analytics: Scope and Challenges , 2008, Visual Data Mining.

[17]  Magdalena Borys,et al.  Eye-tracking metrics in perception and visual attention research , 2017 .

[18]  John Stasko,et al.  BEST PAPER: A Knowledge Task-Based Framework for Design and Evaluation of Information Visualizations , 2004 .

[19]  B. Schmitz,et al.  Berührpunkte mit der Visualisierung , 2013 .

[20]  Alexander Felfernig,et al.  Toward the Next Generation of Recommender Systems: Applications and Research Challenges , 2013 .

[21]  Jian Pei,et al.  Data Mining : Concepts and Techniques 3rd edition Ed. 3 , 2011 .

[22]  Daniel Weiskopf,et al.  Ubiquitous Gaze Sensing and Interaction (Dagstuhl Seminar 18252) , 2018, Dagstuhl Reports.

[23]  Silvia Miksch,et al.  Characterizing Guidance in Visual Analytics , 2017, IEEE Transactions on Visualization and Computer Graphics.

[24]  Martin Raubal,et al.  Improving map reading with gaze-adaptive legends , 2018, ETRA.

[25]  Thies Pfeiffer,et al.  Advantages of eye-gaze over head-gaze-based selection in virtual and augmented reality under varying field of views , 2018, COGAIN@ETRA.

[26]  Pearl Pu,et al.  Consumer decision patterns through eye gaze analysis , 2010, EGIHMI '10.

[27]  Ross T. Smith,et al.  Situated Analytics: Demonstrating immersive analytical tools with Augmented Reality , 2016, J. Vis. Lang. Comput..

[28]  Stelios Sergis,et al.  Increasing Fault Tolerance in Operational Centres using Human Sensing Technologies: Approach and Initial Results , 2015 .

[29]  Tamara Munzner,et al.  A Multi-Level Typology of Abstract Visualization Tasks , 2013, IEEE Transactions on Visualization and Computer Graphics.

[30]  Roman Bednarik,et al.  Potentials of Eye-Movement Tracking in Adaptive Systems , 2009 .

[31]  Thies Pfeiffer,et al.  Attention guiding techniques using peripheral vision and eye tracking for feedback in augmented-reality-based assistance systems , 2017, 2017 IEEE Symposium on 3D User Interfaces (3DUI).

[32]  Kristin A. Cook,et al.  Illuminating the Path: The Research and Development Agenda for Visual Analytics , 2005 .

[33]  Jing Yang,et al.  Guidance in the human-machine analytics process , 2018, Vis. Informatics.

[34]  Radu Jianu,et al.  A Gaze‐enabled Graph Visualization to Improve Graph Reading Tasks , 2014, Comput. Graph. Forum.

[35]  Dieter W. Fellner,et al.  Leveraging eye-gaze and time-series features to predict user interests and build a recommendation model for visual analysis , 2018, ETRA.

[36]  Begoña García Zapirain,et al.  Assessing Visual Attention Using Eye Tracking Sensors in Intelligent Cognitive Therapies Based on Serious Games , 2015, Sensors.

[37]  Christophe Hurter,et al.  FiberClay: Sculpting Three Dimensional Trajectories to Reveal Structural Insights , 2019, IEEE Transactions on Visualization and Computer Graphics.

[38]  Radu Jianu,et al.  Analyzing Eye-Tracking Information in Visualization and Data Space: From Where on the Screen to What on the Screen , 2017, IEEE Transactions on Visualization and Computer Graphics.

[39]  John T. Stasko,et al.  Toward a Deeper Understanding of the Role of Interaction in Information Visualization , 2007, IEEE Transactions on Visualization and Computer Graphics.

[40]  Michael Burch,et al.  Visualization of Eye Tracking Data: A Taxonomy and Survey , 2017, Comput. Graph. Forum.

[41]  Arjan Kuijper,et al.  Interaction Taxonomy for Tracking of User Actions in Visual Analytics Applications , 2014, Handbook of Human Centric Visualization.

[42]  Daniel A. Keim,et al.  Visual Analytics of Movement , 2013, Springer Berlin Heidelberg.

[43]  Thomas Ertl,et al.  VA2: A Visual Analytics Approach for Evaluating Visual Analytics Applications , 2016, IEEE Transactions on Visualization and Computer Graphics.

[44]  Pearl Pu,et al.  Eye-tracking product recommenders' usage , 2010, RecSys '10.

[45]  David O'Sullivan,et al.  Geographic Information Analysis , 2002 .

[46]  Ben Shneiderman,et al.  The eyes have it: a task by data type taxonomy for information visualizations , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.

[47]  Hao Jiang,et al.  Personalized online document, image and video recommendation via commodity eye-tracking , 2008, RecSys '08.

[48]  Tobias Schreck,et al.  Visual Exploration of Large Scatter Plot Matrices by Pattern Recommendation based on Eye Tracking , 2017, ESIDA@IUI.

[49]  R. Brunet.  Bertin (Jacques). Sémiologie graphique. Paris, Mouton et Gauthier- Villars, 1967 , 1968 .

[50]  Jiawei Han,et al.  Data Mining: Concepts and Techniques , 2000 .

[51]  Martin Raubal,et al.  Using eye movements to recognize activities on cartographic maps , 2013, SIGSPATIAL/GIS.

[52]  Alex Endert,et al.  The State of the Art in Integrating Machine Learning into Visual Analytics , 2017, Comput. Graph. Forum.

[53]  Jose Luis Perez Velazquez,et al.  Eye Movement Measurement in Diagnostic Assessment of Disorders of Consciousness , 2014, Front. Neurol..

[54]  Li Chen,et al.  Users' eye gaze pattern in organization-based recommender interfaces , 2011, IUI '11.

[55]  Jian Pei,et al.  Data Mining: Concepts and Techniques, 3rd edition , 2006 .

[56]  Nammee Moon,et al.  A Preference Based Recommendation System Design Through Eye-Tracking and Social Behavior Analysis , 2017, CSA/CUTE.

[57]  Daniel A. Keim,et al.  Solving Problems with Visual Analytics , 2011, FET.

[58]  M A Just,et al.  A theory of reading: from eye fixations to comprehension. , 1980, Psychological review.

[59]  David S. Ebert,et al.  A Mobile Visual Analytics Approach for Law Enforcement Situation Awareness , 2014, 2014 IEEE Pacific Visualization Symposium.

[60]  Michael Burch,et al.  Visual Analytics Methodology for Eye Movement Studies , 2012, IEEE Transactions on Visualization and Computer Graphics.

[61]  Daniel A. Keim,et al.  Mastering the Information Age - Solving Problems with Visual Analytics , 2010 .

[62]  Dieter W. Fellner,et al.  Visual Exploration of Hierarchical Data Using Degree-of-Interest Controlled by Eye-Tracking , 2016, FMT.

[63]  Bernhard Preim,et al.  Interaktive Systeme - Band 1: Grundlagen, Graphical User Interfaces, Informationsvisualisierung , 2010 .