Towards a systematic understanding of graphical cues in communication through statistical graphs

Statistical graphs-in particular, line graphs and bar graphs-are efficient means of communication in a wide range of non-expert settings. In communication settings, statistical graphs do not only serve as visualizations of individual data points but also provide visual access to various aspects of the information contained in data. Moreover, specific types of graphs are better means for providing visual access to certain aspects of data. For instance, trend information is visually conveyed through line graphs and bar graphs in the time domain. The interpretation of the information content in a graph is influenced by several factors, such as perceptual salience of line segments in a line graph. In addition, the presence of graphical cues substantially influences the interpretation of graph readers. Graphical cues are visual elements, usually in the form of point markers, non-directional lines, curves and arrows. They play a communicative role in communication through graphs. The present study reports an experimental investigation, in which the participants provided verbal descriptions of a set of graphs with/without graphical cues. The stimuli involved line graphs and bar graphs that represented the same data. The analyses of eye movements and verbal protocols reveal that the interpretations of the participants are systematically influenced by the presence or absence of a graphical cue, the type of the graphical cue (i.e., a point marker vs. an arrow), as well as the type of the graph (i.e., a line graph vs. a bar graph).

[1]  John Beavers,et al.  Scalar complexity and the structure of events , 2008 .

[2]  Mary Hegarty,et al.  The Cognitive Science of Visual-Spatial Displays: Implications for Design , 2011, Top. Cogn. Sci..

[3]  Richard K. Lowe The Cambridge Handbook of Multimedia Learning: Multimedia Learning of Meteorology , 2005 .

[4]  M. Tanenhaus,et al.  Approaches to studying world-situated language use : bridging the language-as-product and language-as-action traditions , 2005 .

[5]  Cengiz Acartürk,et al.  Multimodal comprehension of graph-text constellations: an information processing perspective , 2010 .

[6]  Lars Kulik,et al.  Shape Nouns and Shape Concepts: A Geometry for 'Corner' , 1998, Spatial Cognition.

[7]  Lars Kulik,et al.  Lexical Specifications of Paths , 2000, Spatial Cognition.

[8]  Jeffrey V. Nickerson,et al.  Representing Category and Continuum: Visualizing Thought , 2012, Diagrams.

[9]  Christopher Habel,et al.  Abstract Structures in Spatial Cognition , 1997, Foundations of Computer Science: Potential - Theory - Cognition.

[10]  Peter C.-H. Cheng,et al.  Extending task analytic models of graph-based reasoning: A cognitive model of problem solving with Cartesian graphs in ACT-R/PM , 2002, Cognitive Systems Research.

[11]  Jeffrey Heer,et al.  Design Considerations for Collaborative Visual Analytics , 2008, Inf. Vis..

[12]  C. Habel,et al.  Eye Tracking in Multimodal Comprehension of Graphs , 2012 .

[13]  Laura A. Carlson,et al.  Functional Features in Language and Space. , 2004 .

[14]  Cengiz Acartürk,et al.  Points, Lines and Arrows in Statistical Graphs , 2012, Diagrams.

[15]  Cengiz Acartürk,et al.  Multimodal Comprehension of Graphics with Textual Annotations: The Role of Graphical Means Relating Annotations and Graph Lines , 2008, Diagrams.

[16]  Christopher Habel,et al.  On Reciprocal Improvement in Multimodal Generation : Co-reference by Text and Information Graphics * , 2007 .

[17]  Patricia D. Mautone,et al.  Cognitive Aids for Guiding Graph Comprehension. , 2007 .

[18]  F. Attneave Some informational aspects of visual perception. , 1954, Psychological review.

[19]  Henk J. Verkuyl,et al.  On the Compositional Nature of the Aspects , 1972 .

[20]  J. Corter,et al.  The Process of Probability Problem Solving: Use of External Visual Representations , 2010 .

[21]  Maneesh Agrawala,et al.  Perceptual interpretation of ink annotations on line charts , 2009, UIST '09.

[22]  Gerald L. Lohse,et al.  A Cognitive Model for Understanding Graphical Perception , 1993, Hum. Comput. Interact..

[23]  James D. Hollan,et al.  Deixis and the future of visualization excellence , 1991, Proceeding Visualization '91.

[24]  Barbara Tversky,et al.  Graphs in Print , 2002, Diagrammatic Representation and Reasoning.

[25]  Barbara Tversky,et al.  Arrows in Comprehending and Producing Mechanical Diagrams , 2006, Cogn. Sci..

[26]  Z. Harris,et al.  Foundations of language , 1941 .

[27]  R McGill,et al.  Graphical Perception and Graphical Methods for Analyzing Scientific Data , 1985, Science.

[28]  Alfred Bork,et al.  Multimedia in Learning , 2001 .

[29]  Patrick Olivier,et al.  Diagrammatic Representation and Reasoning , 2001 .

[30]  M. Hegarty,et al.  Graphs as aids to knowledge construction: Signaling techniques for guiding the process of graph comprehension. , 1999 .

[31]  Herbert Freeman,et al.  Shape description via the use of critical points , 1978, Pattern Recognit..

[32]  M. Tanenhaus,et al.  Introduction to the special issue on language–vision interactions , 2007 .

[33]  Christopher Habel,et al.  Causal Inference in Graph-Text Constellations: Designing Verbally Annotated Graphs**Supported in part by DFG (German Science Foundation) in ITRG 1247 ‘Cross-modal Interaction in Natural and Artificial Cognitive Systems’ (CI-NACS) , 2011 .

[34]  R. Mayer,et al.  Multimedia Learning: Frontmatter , 2001 .

[35]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[36]  P. Schumacher,et al.  Constraints on ontology changing complexation processes: Evidence from event-related brain potentials , 2010 .

[37]  H. H. Clark Pointing and placing. , 2003 .

[38]  Zenon Kulpa,et al.  DIAGRAMMATIC REPRESENTATION AND REASONING , 1994 .

[39]  J. Feldman,et al.  Information along contours and object boundaries. , 2005, Psychological review.

[40]  Lars Kulik,et al.  A Geometric Agent Following Route Instructions , 2003, Spatial Cognition.

[41]  Robert P. Futrelle Ambiguity in visual language theory and its role in diagram parsing , 1999, Proceedings 1999 IEEE Symposium on Visual Languages.

[42]  Jeffery. M. Zacks,et al.  Bars and lines: A study of graphic communication , 1999, Memory & cognition.

[43]  Wilfried Brauer,et al.  Foundations of computer science : potential--theory--cognition , 1997 .

[44]  R. Jackendoff The architecture of the linguistic-spatial interface , 1996 .

[45]  Paul U. Lee,et al.  Lines, Blobs, Crosses and Arrows: Diagrammatic Communication with Schematic Figures , 2000, Diagrams.

[46]  Richard K. Lowe,et al.  An Eye Tracking Comparison of External Pointing Cues and Internal Continuous Cues in Learning with Complex Animations , 2010 .

[47]  M. Malbrán The Cambridge Handbook of Multimedia Learning , 2007 .

[48]  S. Kosslyn Understanding charts and graphs , 1989 .

[49]  Steven Pinker,et al.  A theory of graph comprehension. , 1990 .

[50]  Bill Winn,et al.  Charts, Graphs, and Diagrams in Educational Materials , 1987 .

[51]  Zeno Vendler,et al.  Verbs and Times , 1957, The Language of Time - A Reader.

[52]  Teenie Matlock,et al.  The integration of figurative language and static depictions: An eye movement study of fictive motion , 2007, Cognition.

[53]  Herbert A. Simon,et al.  Why a Diagram is (Sometimes) Worth Ten Thousand Words , 1987, Cogn. Sci..

[54]  C. Acartürk Referring expressions in communication through line graphs : A comparative analysis of verbal descriptions , 2015 .

[55]  S. Brennan 4 How Conversation Is Shaped by Visual and Spoken Evidence , 2005 .

[56]  R. Jackendoff Foundations of Language: Brain, Meaning, Grammar, Evolution , 2002 .

[57]  Cengiz Acartürk,et al.  Gestures in Communication through Line Graphs , 2012, CogSci.

[58]  S. Kosslyn Image and mind , 1982 .

[59]  Richard Mayer,et al.  Multimedia Learning , 2001, Visible Learning Guide to Student Achievement.

[60]  Richard Lowe,et al.  Cueing complex animations: Does direction of attention foster learning processes? , 2011 .

[61]  Manfred Consten,et al.  9. Complex anaphors in discourse , 2008 .