Understand users’ comprehension and preferences for composing information visualizations

We are developing an automated visualization system that helps users combine two or more existing information graphics to form an integrated view. To establish empirical foundations for building such a system, we designed and conducted two studies on Amazon Mechanical Turk to understand users’ comprehension and preferences of composite visualization under different conditions (e.g., data and tasks). In Study 1, we collected more than 1,500 textual descriptions capturing about 500 participants’ insights of given information graphics, which resulted in a task-oriented taxonomy of visual insights. In Study 2, we asked 240 participants to rank composite visualizations by their suitability for acquiring a given visual insight identified in Study 1, which resulted in ranked user preferences of visual compositions for acquiring each type of insight. In this article, we report the details of our two studies and discuss the broader implications of our crowdsourced research methodology and results to HCI-driven visualization research.

[1]  Steven Pinker,et al.  A theory of graph comprehension. , 1990 .

[2]  Christopher Rao,et al.  Graphs in Statistical Analysis , 2010 .

[3]  Michelle X. Zhou,et al.  Evaluating the Use of Data Transformation for Information Visualization , 2008, IEEE Transactions on Visualization and Computer Graphics.

[4]  Steven F. Roth,et al.  On the semantics of interactive visualizations , 1996, Proceedings IEEE Symposium on Information Visualization '96.

[5]  R McGill,et al.  Graphical Perception and Graphical Methods for Analyzing Scientific Data , 1985, Science.

[6]  Charles J. Fillmore,et al.  Frames and the semantics of understanding , 1985 .

[7]  Priti Shah,et al.  A Model of the Perceptual and Conceptual Processes in Graph Comprehension , 1998 .

[8]  Niklas Elmqvist,et al.  Exploring the design space of composite visualization , 2012, 2012 IEEE Pacific Visualization Symposium.

[9]  Eric R. Ziegel,et al.  Generalized Linear Models , 2002, Technometrics.

[10]  George Kingsley Zipf,et al.  Human behavior and the principle of least effort , 1949 .

[11]  Hadley Wickham,et al.  ggplot2 - Elegant Graphics for Data Analysis (2nd Edition) , 2017 .

[12]  Sung-Hee Kim,et al.  How to filter out random clickers in a crowdsourcing-based study? , 2012, BELIV '12.

[13]  Ben Shneiderman,et al.  The eyes have it: a task by data type taxonomy for information visualizations , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.

[14]  R. Burt,et al.  THE GENERAL SOCIAL SURVEY * , 2003 .

[15]  Evelyn Goldsmith.  Research into Illustration:An Approach and a Review , 1984 .

[16]  Leland Wilkinson,et al.  The Grammar of Graphics (Statistics and Computing) , 2005 .

[17]  M. Friedman.  The Use of Ranks to Avoid the Assumption of Normality Implicit in the Analysis of Variance , 1937 .

[18]  Steven K. Feiner,et al.  Visual task characterization for automated visual discourse synthesis , 1998, CHI.

[19]  Richard W. Hamming,et al.  Error detecting and error correcting codes , 1950 .

[20]  Michelle X. Zhou,et al.  An optimization-based approach to dynamic data transformation for smart visualization , 2008, IUI '08.

[21]  G. Lakoff,et al.  Metaphors We Live by , 1982 .

[22]  David K. Simkin,et al.  An Information-Processing Analysis of Graph Perception , 1987 .

[23]  M. Sheelagh T. Carpendale,et al.  VisGets: Coordinated Visualizations for Web-based Information Exploration and Discovery , 2008, IEEE Transactions on Visualization and Computer Graphics.

[24]  Martin Wattenberg,et al.  ManyEyes: a Site for Visualization at Internet Scale , 2007, IEEE Transactions on Visualization and Computer Graphics.

[25]  Priti Shah,et al.  Bar and Line Graph Comprehension: An Interaction of Top-Down and Bottom-Up Processes , 2011, Top. Cogn. Sci..

[26]  G. Fauconnier,et al.  The Way We Think: Conceptual Blending and the Mind''s Hidden Complexities. Basic Books , 2002 .

[27]  David D. Woods,et al.  Visual Momentum: A Concept to Improve the Cognitive Coupling of Person and Computer , 1984, Int. J. Man Mach. Stud..

[28]  Steven F. Roth,et al.  Automating the presentation of information , 1991, [1991] Proceedings. The Seventh IEEE Conference on Artificial Intelligence Application.

[29]  Steven K. Feiner,et al.  Data characterization for automatically visualizing heterogeneous information , 1996, Proceedings IEEE Symposium on Information Visualization '96.

[30]  M. Hegarty,et al.  Graphs as aids to knowledge construction: Signaling techniques for guiding the process of graph comprehension. , 1999 .

[31]  James R. Eagan,et al.  Low-level components of analytic activity in information visualization , 2005, IEEE Symposium on Information Visualization, 2005. INFOVIS 2005..

[32]  Susan N. Friel,et al.  Making sense of graphs: Critical factors influencing comprehension and instructional implications. , 2001 .

[33]  Edward Rolf Tufte,et al.  The visual display of quantitative information , 1985 .

[34]  Douglas B. Downey,et al.  WHEN BIGGER IS NOT BETTER: FAMILY SIZE, PARENTAL RESOURCES, AND CHILDREN'S EDUCATIONAL PERFORMANCE* , 1995 .

[35]  Martin Wattenberg,et al.  Your place or mine?: visualization as a community component , 2008, CHI.

[36]  L. Barsalou,et al.  Whither structured representation? , 1999, Behavioral and Brain Sciences.

[37]  E. Kaneko,et al.  A study on the improved tanker structure against collision and grounding damage , 1998 .

[38]  L. Talmy.  Toward a Cognitive Semantics , 2003 .

[39]  Rebecca M. Blank,et al.  Race and gender in the labor market , 1999 .

[40]  Priti Shah,et al.  The Comprehension of Quantitative Information in Graphical Displays. , 2005 .

[41]  Klaus Krippendorff,et al.  Answering the Call for a Standard Reliability Measure for Coding Data , 2007 .

[42]  Stephen M. Casner,et al.  Task-analytic approach to the automated design of graphic presentations , 1991, TOGS.

[43]  Adam J. Berinsky,et al.  Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk , 2012, Political Analysis.

[44]  X ZhouMichelle,et al.  Understand users comprehension and preferences for composing information visualizations , 2014 .

[45]  Steven F. Roth,et al.  Toward an Information Visualization Workspace: Combining Multiple Means of Expression , 1997, Hum. Comput. Interact..

[46]  P. McCullagh,et al.  Generalized Linear Models , 1992 .

[47]  A. Strauss,et al.  The discovery of grounded theory: strategies for qualitative research aldine de gruyter , 1968 .

[48]  Richard A. Becker,et al.  The Visual Design and Control of Trellis Display , 1996 .

[49]  K. Gegenfurtner,et al.  Design Issues in Gaze Guidance Under review with ACM Transactions on Computer Human Interaction , 2009 .

[50]  Z. Kövecses,et al.  Metonymy: Developing a cognitive linguistic view , 1998 .

[51]  D. Eckberg,et al.  Gender and environmentalism : Results from the 1993 General Social Survey : Research on the environment , 1997 .

[52]  Mark Bailey,et al.  The Grammar of Graphics , 2007, Technometrics.

[53]  Min Chen,et al.  Automated Generation of Graphic Sketches by Example , 2003, IJCAI.

[54]  Michelle X. Zhou,et al.  An optimization-based approach to dynamic visual context management , 2005, IEEE Symposium on Information Visualization, 2005. INFOVIS 2005..

[55]  George Lakoff,et al.  Women, Fire, and Dangerous Things , 1987 .

[56]  S. Kosslyn.  Understanding charts and graphs , 1989 .

[57]  Mary Czerwinski,et al.  Empirical evaluation of information visualizations: an introduction , 2000, Int. J. Hum. Comput. Stud..

[58]  N. Presmeg.  The body in the mind: The bodily basis of meaning, imagination and reason , 1992 .

[59]  Kristin A. Cook,et al.  Illuminating the Path: The Research and Development Agenda for Visual Analytics , 2005 .

[60]  Frances R. Curcio.  Developing Graph Comprehension: Elementary and Middle School Activities , 1989 .

[61]  Chris North,et al.  Toward measuring visualization insight , 2006, IEEE Computer Graphics and Applications.

[62]  A. Adithya.  Parallel Coordinates , 2015 .

[63]  Mary Hegarty,et al.  Effects of knowledge and display design on comprehension of complex graphics , 2010 .

[64]  P. Gärdenfors,et al.  Cognitive semantics : meaning and cognition , 1999 .

[65]  Zhen Wen,et al.  Behavior-driven visualization recommendation , 2009, IUI.

[66]  Jock D. Mackinlay,et al.  Automating the design of graphical presentations of relational information , 1986, TOGS.

[67]  Jeffrey Heer,et al.  Crowdsourcing graphical perception: using mechanical turk to assess visualization design , 2010, CHI.

[68]  Edward R. Tufte,et al.  The Visual Display of Quantitative Information , 1986 .

[69]  John J. Bertin,et al.  The semiology of graphics , 1983 .

[70]  Mary Czerwinski,et al.  Understanding the verbal language and structure of end-user descriptions of data visualizations , 2012, CHI.

[71]  Martin Wattenberg,et al.  Voyagers and voyeurs: supporting asynchronous collaborative information visualization , 2007, CHI.

[72]  John T. Stasko,et al.  An evaluation of space-filling information visualizations for depicting hierarchical structures , 2000, Int. J. Hum. Comput. Stud..

[73]  Martin A. Andresen,et al.  Obesity relationships with community design, physical activity, and time spent in cars. , 2004, American journal of preventive medicine.

[74]  Jeffrey Heer,et al.  Sizing the horizon: the effects of chart size and layering on the graphical perception of time series visualizations , 2009, CHI.

[75]  Jean Lemaire,et al.  Automobile Insurance: Actuarial Models , 1985 .

[76]  Vyvyan Evans,et al.  Cognitive Linguistics: An Introduction , 2001 .

[77]  Pat Hanrahan,et al.  Show Me: Automatic Presentation for Visual Analysis , 2007, IEEE Transactions on Visualization and Computer Graphics.

[78]  M. Sheelagh T. Carpendale,et al.  VisLink: Revealing Relationships Amongst Visualizations , 2007, IEEE Transactions on Visualization and Computer Graphics.

[79]  Richard D. Powers,et al.  A study of graph comprehension difficulties , 1959 .

[80]  Panagiotis G. Ipeirotis.  Demographics of Mechanical Turk , 2010 .

[81]  Rebecca M. Blank,et al.  Chapter 48 Race and gender in the labor market , 1999 .

[82]  David Gillborn,et al.  Educational Inequality: Mapping Race, Class and Gender. A Synthesis of Research Evidence. , 2000 .

[83]  Gerardine DeSanctis,et al.  COMPUTER GRAPHICS AS DECISION AIDS: DIRECTIONS FOR RESEARCH* , 1984 .

[84]  Gilles Fauconnier,et al.  Mental Spaces: Aspects of Meaning Construction in Natural Language , 1985 .

[85]  Jeffrey Heer,et al.  Animated Transitions in Statistical Data Graphics , 2007, IEEE Transactions on Visualization and Computer Graphics.

[86]  Hadley Wickham,et al.  Product Plots , 2011, IEEE Transactions on Visualization and Computer Graphics.