Navigating reductionism and holism in evaluation

In this position paper, we enumerate two approaches to the evaluation of visualizations which are associated with two approaches to knowledge formation in science: reductionism, which holds that the understanding of complex phenomena is based on the understanding of simpler components; and holism, which states that complex phenomena have characteristics more than the sum of their parts and must be understood as complete, irreducible units. While we believe that each approach has benefits for evaluating visualizations, we claim that strict adherence to one perspective or the other can make it difficult to generate a full evaluative picture of visualization tools and techniques. We argue for movement between and among these perspectives in order to generate knowledge that is both grounded (i.e. its constituent parts work) and validated (i.e. the whole operates correctly). We conclude with examples of techniques which we believe represent movements of this sort from our own work, highlighting areas where we have both "built up" reductionist techniques into larger contexts, and "broken down" holistic techniques to create generalizable knowledge.

[1]  Tobias Isenberg,et al.  A Systematic Review on the Practice of Evaluating Visualization , 2013, IEEE Transactions on Visualization and Computer Graphics.

[2]  Danielle Albers Szafir,et al.  Lightness Constancy in Surface Visualization , 2016, IEEE Transactions on Visualization and Computer Graphics.

[3]  Jarke J. van Wijk,et al.  The value of visualization , 2005, VIS 05. IEEE Visualization, 2005..

[4]  Michael Gleicher,et al.  Quantity estimation in visualizations of tagged text , 2013, CHI.

[5]  Michael Gleicher,et al.  Serendip: Topic model-driven visual exploration of text corpora , 2014, 2014 IEEE Conference on Visual Analytics Science and Technology (VAST).

[6]  Michael Gleicher,et al.  Task-driven evaluation of aggregation in time series visualization , 2014, CHI.

[7]  Michael Gleicher,et al.  Molecular Surface Abstraction , 2007, IEEE Transactions on Visualization and Computer Graphics.

[8]  Ben Shneiderman,et al.  Strategies for evaluating information visualization tools: multi-dimensional in-depth long-term case studies , 2006, BELIV '06.

[9]  Chris North,et al.  Toward measuring visualization insight , 2006, IEEE Computer Graphics and Applications.

[10]  Michael Gleicher,et al.  Sequence Surveyor: Leveraging Overview for Scalable Genomic Alignment Visualization , 2011, IEEE Transactions on Visualization and Computer Graphics.

[11]  E. L. Kaufman,et al.  The discrimination of visual number. , 1949, The American journal of psychology.

[12]  Michael Gleicher,et al.  Local functional descriptors for surface comparison based binding prediction , 2012, BMC Bioinformatics.

[13]  Robert McGill,et al.  An Experiment in Graphical Perception , 1986, Int. J. Man Mach. Stud..

[14]  Steven Franconeri,et al.  Comparing averages in time series data , 2012, CHI.

[15]  Tamara Munzner,et al.  The four-level nested model revisited: blocks and guidelines , 2012, BELIV '12.

[16]  Michael Gleicher,et al.  Visualizing Validation of Protein Surface Classifiers , 2014, Comput. Graph. Forum.

[17]  M. Sheelagh T. Carpendale,et al.  Empirical Studies in Information Visualization: Seven Scenarios , 2012, IEEE Transactions on Visualization and Computer Graphics.

[18]  W. Cleveland,et al.  Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods , 1984 .