Analyzing gaze behavior for text-embellished narrative visualizations under different task scenarios

Abstract We conduct an eye tracking study to investigate perception text-embellished narrative visualizations under different task conditions. Study stimuli are data visualizations embellished with text-based elements: annotations, captions, labels, and descriptive text. We consider three common viewing tasks that occur when these types of graphics are viewed: (1) simple observation, (2) active search to answer a query, and (3) information memorization for later recall. The overarching goal is to understand, at a perceptual level, if and how task affects how these visualizations are interacted with. By analyzing collected gaze data and conducting advanced semantic scanpath analysis, we find, at a high level, diverse patterns of gaze behavior: simple observation and information memorization lead to similar optical viewing strategies, while active search significantly diverges, both in regards to which areas of the visualization are focused upon and how often embellishments are interacted with. We discuss study outcomes in the context of embellishing visualizations with text for various usage scenarios.

[1]  Donald D. Hoffman,et al.  Salience of visual parts , 1997, Cognition.

[2]  Ulric Neisser,et al.  The Control of Information Pickup in Selective Looking , 2019, Perception and its Development.

[3]  Sarah Cohen,et al.  Computational journalism , 2011, Commun. ACM.

[4]  Michael Burch,et al.  User performance and reading strategies for metro maps: An eye tracking study , 2017, Spatial Cogn. Comput..

[5]  Jeffrey Heer,et al.  Narrative Visualization: Telling Stories with Data , 2010, IEEE Transactions on Visualization and Computer Graphics.

[6]  K. Rayner Eye movements in reading and information processing: 20 years of research. , 1998, Psychological bulletin.

[7]  Cole Nussbaumer Knaflic,et al.  Storytelling with Data: A Data Visualization Guide for Business Professionals , 2015 .

[8]  Irene Reppa,et al.  An Empirical Study on Using Visual Embellishments in Visualization , 2012, IEEE Transactions on Visualization and Computer Graphics.

[9]  Hanspeter Pfister,et al.  What Makes a Visualization Memorable? , 2013, IEEE Transactions on Visualization and Computer Graphics.

[10]  Michael J. Haass,et al.  Data Visualization Saliency Model: A Tool for Evaluating Abstract Data Visualizations , 2018, IEEE Transactions on Visualization and Computer Graphics.

[11]  W. Cleveland,et al.  Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods , 1984 .

[12]  Patrick J. F. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 2003 .

[13]  Ivan Viola,et al.  Exploring visual attention and saliency modeling for task-based visual analysis , 2018, Comput. Graph..

[14]  Irene Reppa,et al.  Visual Aesthetic Appeal Speeds Processing of Complex but not Simple Icons , 2008 .

[15]  Frédo Durand,et al.  Learning Visual Importance for Graphic Designs and Data Visualizations , 2017, UIST.

[16]  Kwan-Liu Ma,et al.  Scientific Storytelling Using Visualization , 2012, IEEE Computer Graphics and Applications.

[17]  James T. Enns,et al.  Attention and Visual Memory in Visualization and Computer Graphics , 2012, IEEE Transactions on Visualization and Computer Graphics.

[18]  Cristina Conati,et al.  User-adaptive information visualization: using eye gaze data to infer visualization tasks and user cognitive abilities , 2013, IUI '13.

[19]  M. Just,et al.  Constructing mental models of machines from text and diagrams. , 1993 .

[20]  R. Mayer,et al.  A generative theory of textbook design: Using annotated illustrations to foster meaningful learning of science text , 1995 .

[21]  P. Shah,et al.  Review of Graph Comprehension Research: Implications for Instruction , 2002 .

[22]  Evan M. Peck,et al.  The Curious Case of Combining Text and Visualization , 2019, EuroVis.

[23]  Cengiz Acartürk,et al.  Points, Lines and Arrows in Statistical Graphs , 2012, Diagrams.

[24]  S. B. Needleman,et al.  A general method applicable to the search for similarities in the amino acid sequence of two proteins. , 1970, Journal of molecular biology.

[25]  J. Wolfe,et al.  Guided Search 2.0 A revised model of visual search , 1994, Psychonomic bulletin & review.

[26]  A. L. I︠A︡rbus Eye Movements and Vision , 1967 .

[27]  K. Rayner The 35th Sir Frederick Bartlett Lecture: Eye movements and attention in reading, scene perception, and visual search , 2009, Quarterly journal of experimental psychology.

[28]  Denis Drieghe,et al.  Parafoveal-on-foveal effects on eye movements during reading , 2011 .

[29]  Carl Gutwin,et al.  Useful junk?: the effects of visual embellishment on comprehension and memorability of charts , 2010, CHI.

[30]  Hansjörg Schmauder,et al.  An Evaluation of Visual Search Support in Maps , 2017, IEEE Transactions on Visualization and Computer Graphics.

[31]  Nahum Gershon,et al.  What storytelling can do for information visualization , 2001, Commun. ACM.

[32]  Mary Hegarty,et al.  When static media promote active learning: annotated illustrations versus narrated animations in multimedia instruction. , 2005, Journal of experimental psychology. Applied.

[33]  Andrew Vande Moere,et al.  On the role of design in information visualization , 2011, Inf. Vis..

[34]  Nicholas Diakopoulos,et al.  Visualization Rhetoric: Framing Effects in Narrative Visualization , 2011, IEEE Transactions on Visualization and Computer Graphics.

[35]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.

[36]  Michael Burch,et al.  Interactive scanpath-oriented annotation of fixations , 2016, ETRA.

[37]  Kenneth Holmqvist,et al.  Eye tracking: a comprehensive guide to methods and measures , 2011 .

[38]  Joseph H. Goldberg,et al.  Eye tracking for visualization evaluation: Reading values on linear versus radial graphs , 2011, Inf. Vis..

[39]  Kamran Sedig,et al.  Transactions on Human-computer Interaction Thci Design for Complex Cognitive Activities with Visual Representations: a Pattern-based Approach Sedig and Parsons Interaction Design for Complex Cognitive Activities with Visualizations , 2022 .

[40]  Hanspeter Pfister,et al.  Beyond Memorability: Visualization Recognition and Recall , 2016, IEEE Transactions on Visualization and Computer Graphics.

[41]  Richard C. Atkinson,et al.  Human Memory: A Proposed System and its Control Processes , 1968, Psychology of Learning and Motivation.

[42]  Michael J. Haass,et al.  Patterns of Attention: How Data Visualizations Are Read , 2017, HCI.