Visualizations of binary data: A comparative evaluation

Data visualization has the potential to assist humans in analysing and comprehending large volumes of data, and to detect patterns, clusters and outliers that are not obvious using nongraphical forms of presentation. For this reason, data visualizations have an important role to play in a diverse range of applied problems, including data exploration and mining, information retrieval, and intelligence analysis. Unfortunately, while various different approaches are available for data visualization, there have been few rigorous evaluations of their effectiveness. This paper presents the results of three controlled experiments comparing the ability of four different visualization approaches to help people answer meaningful questions for binary data sets. Two of these visualizations, Chernoff faces and star glyphs, represent objects using simple icon-like displays. The other two visualizations use a spatial arrangement of the objects, based on a model of human mental representation, where more similar objects are placed nearer each other. One of these spatial displays uses a common features model of similarity, while the other uses a distinctive features model. The first experiment finds that both glyph visualizations lead to slow, inaccurate answers being given with low confidence, while the faster and more confident answers for spatial visualizations are only accurate when the common features similarity model is used. The second experiment, which considers only the spatial visualizations, supports this finding, with the common features approach again producing more accurate answers. The third experiment measures human performance using the raw data in tabular form, and so allows the usefulness of visualizations in facilitating human performance to be assessed. This experiment confirms that people are faster, more confident and more accurate when an appropriate visualization of the data is made available.

[1]  George Henry Dunteman,et al.  Introduction To Multivariate Analysis , 1984 .

[2]  Eric R. Ziegel,et al.  Applied Multivariate Data Analysis , 2002, Technometrics.

[3]  I. J. Myung,et al.  Applying Occam’s razor in modeling cognition: A Bayesian approach , 1997 .

[4]  A. Tversky.  Features of Similarity , 1977 .

[5]  Yili Liu,et al.  Use of Computer Graphics and Cluster Analysis in Aiding Relational Judgment , 1989 .

[6]  A. Tversky,et al.  Additive similarity trees , 1977 .

[7]  C. Howson,et al.  Scientific Reasoning: The Bayesian Approach , 1989 .

[8]  T. Louis,et al.  Bayes and Empirical Bayes Methods for Data Analysis. , 1997 .

[9]  James A. Wise,et al.  The Ecological Approach to Text Visualization , 1999, J. Am. Soc. Inf. Sci..

[10]  J. Hunter.  Needed: A Ban on the Significance Test , 1997 .

[11]  J. W. Hutchinson,et al.  Nearest neighbor analysis of psychological spaces. , 1986 .

[12]  John Long,et al.  Target Paper Conception of the cognitive engineering design problem , 1998 .

[13]  Colin Ware,et al.  Information Visualization: Perception for Design , 2000 .

[14]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[15]  Mary Anne Buttigieg,et al.  Object Displays Do Not Always Support Better Integrated Task Performance , 1989 .

[16]  Herman Chernoff,et al.  The Use of Faces to Represent Points in k- Dimensional Space Graphically , 1973 .

[17]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[18]  H. Chernoff,et al.  Effect on Classification Error of Random Permutations of Features in Representing Multivariate Data by Faces , 1975 .

[19]  Clayton Lewis,et al.  A problem-oriented classification of visualization techniques , 1990, Proceedings of the First IEEE Conference on Visualization: Visualization `90.

[20]  J. Gregory Trafton,et al.  Turning pictures into numbers: extracting and generating information from complex visualizations , 2000, Int. J. Hum. Comput. Stud..

[21]  Brian Everitt,et al.  Graphical Techniques for Multivariate Data. , 1978 .

[22]  Patricia M. Jones,et al.  The Display of Multivariate Information: An Experimental Study of an Information Integration Task , 1990 .

[23]  Roger N. Shepard,et al.  Additive clustering: Representation of similarities as combinations of discrete overlapping properties. , 1979 .

[24]  Douglas Vickers,et al.  Psychological approaches to data visualisation , 1998 .

[25]  B. Everitt,et al.  Applied Multivariate Data Analysis. , 1993 .

[26]  R N Shepard,et al.  Multidimensional Scaling, Tree-Fitting, and Clustering , 1980, Science.

[27]  R. Shepard.  Perceptual-cognitive universals as reflections of the world. , 2001, The Behavioral and brain sciences.

[28]  M. Lee.  Determining the Dimensionality of Multidimensional Scaling Representations for Cognitive Modeling. , 2001, Journal of mathematical psychology.

[29]  Stephen A. Gilbert,et al.  Structuring information with mental models: a tour of Boston , 1996, CHI.

[30]  Michael Lewis,et al.  Evaluating visualizations: using a taxonomic guide , 2000, Int. J. Hum. Comput. Stud..

[31]  J. Kruschke,et al.  ALCOVE: an exemplar-based connectionist model of category learning. , 1992, Psychological review.

[32]  Stephen M. Kosslyn,et al.  Elements of graph design , 1993 .

[33]  Jonathan D. Cohen,et al.  Drawing graphs to convey proximity: an incremental arrangement method , 1997, TCHI.

[34]  R. Shepard.  Stimulus and response generalization: A stochastic model relating generalization to distance in psychological space , 1957 .

[35]  Michael E. Tipping,et al.  Feed-forward neural networks and topographic mappings for exploratory data analysis , 1996, Neural Computing & Applications.

[36]  Robert J. K. Jacob,et al.  The Face as a Data Display , 1976 .

[37]  J. Carroll,et al.  Spatial, non-spatial and hybrid models for scaling , 1976 .

[38]  R. Nosofsky.  Attention, similarity, and the identification-categorization relationship. , 1986, Journal of experimental psychology. General.

[39]  Jacob Cohen.  The earth is round (p < .05) , 1994 .

[40]  Helen C. Purchase.  The effects of graph layout , 1998, Proceedings 1998 Australasian Computer Human Interaction Conference. OzCHI'98 (Cat. No.98EX234).

[41]  N. Tractinsky,et al.  What is beautiful is usable , 2000, Interact. Comput..

[42]  Helen C. Purchase,et al.  Effective information visualisation: a study of graph drawing aesthetics and algorithms , 2000, Interact. Comput..

[43]  Amos Tversky,et al.  On the relation between common and distinctive feature models , 1987 .

[44]  David Benyon,et al.  Towards a methodology for developing visualizations , 2000, Int. J. Hum. Comput. Stud..

[45]  Anil K. Jain,et al.  Artificial neural networks for feature extraction and multivariate data projection , 1995, IEEE Trans. Neural Networks.

[46]  Ben Shneiderman,et al.  Designing the user interface - strategies for effective human-computer interaction, 3rd Edition , 1997 .

[47]  Timothy Cribbin,et al.  Mapping semantic information in virtual space: dimensions, variance and individual differences , 2000, Int. J. Hum. Comput. Stud..

[48]  W Richards,et al.  Trajectory Mapping: A New Nonmetric Scaling Technique , 1995, Perception.

[49]  Ben Shneiderman,et al.  Designing the user interface (2nd ed.): strategies for effective human-computer interaction , 1992 .

[50]  Sebastian Thrun,et al.  A Bayesian Multiresolution Independence Test for Continuous Variables , 2001, UAI.

[51]  Bradley P. Carlin,et al.  BAYES AND EMPIRICAL BAYES METHODS FOR DATA ANALYSIS , 1996, Stat. Comput..

[52]  M. A. Porter,et al.  Graphical Exploratory Data Analysis. , 1988 .

[53]  J. Kruskal.  Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis , 1964 .

[54]  R. Shepard,et al.  Toward a universal law of generalization for psychological science. , 1987, Science.

[55]  John S. J. Hsu,et al.  Bayesian Methods: An Analysis for Statisticians and Interdisciplinary Researchers , 1999 .

[56]  Michael D. Lee,et al.  A Simple Method for Generating Additive Clustering Models with Limited Complexity , 2002, Machine Learning.

[57]  Differential weighting of common and distinctive components. , 1990, Journal of experimental psychology. General.

[58]  J Meyer,et al.  Performance with tables and graphs: effects of training and a Visual Search Model , 2000, Ergonomics.

[59]  J. Long,et al.  A conception of the cognitive engineering design problem , 1998 .

[60]  L. Wasserman,et al.  Computing Bayes Factors by Combining Simulation and Asymptotic Approximations , 1997 .

[61]  D CohenJonathan.  Drawing graphs to convey proximity , 1997 .

[62]  Patricia M. Jones.  The display of multivariate information , 1986 .

[63]  Chaomei Chen,et al.  Empirical studies of information visualization: a meta-analysis , 2000, Int. J. Hum. Comput. Stud..

[64]  David Lindley,et al.  Bayesian Statistics, a Review , 1987 .

[65]  Yili Liu,et al.  Use of Computer Graphics and Cluster Analysis in Aiding Relational Judgment , 1992 .

[66]  R. Sibson.  Studies in the Robustness of Multidimensional Scaling: Procrustes Statistics , 1978 .

[67]  David S. Ebert,et al.  Experimental analysis of the effectiveness of features in Chernoff faces , 2000, Applied Imaging Pattern Recognition.

[68]  M. Lee,et al.  Extending the ALCOVE model of category learning to featural stimulus domains , 2002, Psychonomic bulletin & review.

[69]  Ben Shneiderman,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction , 1998 .

[70]  P. Arabie,et al.  Mapclus: A mathematical programming approach to fitting the adclus model , 1980 .

[71]  Peter Urbach,et al.  Scientific Reasoning: The Bayesian Approach , 1989 .

[72]  D. M. Green,et al.  On the prediction of confusion matrices from similarity judgments , 1979 .

[73]  Christopher D. Wickens,et al.  Two- and Three-Dimensional Displays for Aviation: A Theoretical and Empirical Comparison , 1993 .

[74]  Christian P. Robert,et al.  The Bayesian choice , 1994 .

[75]  B. Manly.  Multivariate Statistical Methods : A Primer , 1986 .

[76]  A. Tversky,et al.  Weighting common and distinctive features in perceptual and conceptual judgments , 1984, Cognitive Psychology.