Four types of ensemble coding in data visualizations.

Ensemble coding supports rapid extraction of visual statistics about distributed visual information. Researchers typically study this ability with the goal of drawing conclusions about how such coding extracts information from natural scenes. Here we argue that a second domain can serve as another strong inspiration for understanding ensemble coding: graphs, maps, and other visual presentations of data. Data visualizations allow observers to leverage their ability to perform visual ensemble statistics on distributions of spatial or featural visual information to estimate actual statistics on data. We survey the types of visual statistical tasks that occur within data visualizations across everyday examples, such as scatterplots, and more specialized images, such as weather maps or depictions of patterns in text. We divide these tasks into four categories: identification of sets of values, summarization across those values, segmentation of collections, and estimation of structure. We point to unanswered questions for each category and give examples of such cross-pollination in the current literature. Increased collaboration between the data visualization and perceptual psychology research communities can inspire new solutions to challenges in visualization while simultaneously exposing unsolved problems in perception research.

[1]  A. Treisman,et al.  Representation of statistical properties , 2003, Vision Research.

[2]  D. Burr,et al.  A Visual Sense of Number , 2007, Current Biology.

[3]  W. Köhler Gestalt psychology , 1967 .

[4]  Steven Franconeri,et al.  Ranking Visualizations of Correlation Using Weber's Law , 2014, IEEE Transactions on Visualization and Computer Graphics.

[5]  Niklas Elmqvist,et al.  DataMeadow: A Visual Canvas for Analysis of Large-Scale Multivariate Data , 2007 .

[6]  Jacques Bertin,et al.  Semiology of Graphics - Diagrams, Networks, Maps , 2010 .

[7]  R. Kosara,et al.  Parallel sets: visual analysis of categorical data , 2005, IEEE Symposium on Information Visualization, 2005. INFOVIS 2005..

[8]  Paul F. Bulakowski,et al.  Shared attentional resources for global and local motion processing. , 2007, Journal of vision.

[9]  I. Utochkin,et al.  Parallel averaging of size is possible but range-limited: a reply to Marchant, Simons, and De Fockert. , 2014, Acta psychologica.

[10]  Markus Hadwiger,et al.  Ovis: A Framework for Visual Analysisof Ocean Forecast Ensembles , 2014, IEEE Transactions on Visualization and Computer Graphics.

[11]  U. Neisser VISUAL SEARCH. , 1964, Scientific American.

[12]  Tamara Munzner,et al.  MizBee: A Multiscale Synteny Browser , 2009, IEEE Transactions on Visualization and Computer Graphics.

[13]  Niklas Elmqvist,et al.  Graphical Perception of Multiple Time Series , 2010, IEEE Transactions on Visualization and Computer Graphics.

[14]  D. Ariely Seeing Sets: Representation by Statistical Properties , 2001, Psychological science.

[15]  William R. Uttal,et al.  Complexity effects in form detection , 1977, Vision Research.

[16]  D. Levi,et al.  Visual crowding: a fundamental limit on conscious perception and object recognition , 2011, Trends in Cognitive Sciences.

[17]  Jean-Daniel Fekete,et al.  NodeTrix: a Hybrid Visualization of Social Networks , 2007, IEEE Transactions on Visualization and Computer Graphics.

[18]  Dale S. Klopfer,et al.  The perception of scatterplots , 2007, Perception & psychophysics.

[19]  Andrey Chetverikov,et al.  History effects in visual search for monsters: Search times, choice biases, and liking , 2015, Attention, perception & psychophysics.

[20]  Heidrun Schumann,et al.  Visual Methods for Analyzing Time-Oriented Data , 2008, IEEE Transactions on Visualization and Computer Graphics.

[21]  Colin Ware,et al.  Information Visualization: Perception for Design , 2000 .

[22]  Colin Ware,et al.  Visual Thinking for Design , 2008 .

[23]  W. Cleveland,et al.  Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods , 1984 .

[24]  Clayton Lewis,et al.  A problem-oriented classification of visualization techniques , 1990, Proceedings of the First IEEE Conference on Visualization: Visualization `90.

[25]  L. Itti Author address: , 1999 .

[26]  B. Bauer Does Stevens’s Power Law for Brightness Extend to Perceptual Brightness Averaging? , 2009 .

[27]  Gennady Andrienko,et al.  Data and Task Characteristics in Design of Spatio-Temporal Data Visualization Tools , 2002 .

[28]  Ben Shneiderman,et al.  Exploring Data Distributions: Visual Design and Evaluation , 2013, Int. J. Hum. Comput. Interact..

[29]  Steven K. Feiner,et al.  Visual task characterization for automated visual discourse synthesis , 1998, CHI.

[30]  J. Lund,et al.  Compulsory averaging of crowded orientation signals in human vision , 2001, Nature Neuroscience.

[31]  G. Alvarez,et al.  Number estimation relies on a set of segmented objects , 2009, Cognition.

[32]  Jacqui Lee Schiff,et al.  Frames of Reference , 1975 .

[33]  Tony Huzzard,et al.  Towards a Conceptual Framework , 2014 .

[34]  D. Pelli,et al.  Crowding is unlike ordinary masking : Distinguishing feature detection and integration , 2001 .

[35]  J. G. Hollands,et al.  Comparing 2D and 3D Displays for Trend Estimation: The Effects of Display Augmentation , 1999 .

[36]  David R. Brillinger,et al.  Time Series , 2018, Randomization, Bootstrap and Monte Carlo Methods in Biology.

[37]  Stacie Hibino Task Analysis for Information Visualization , 1999, VISUAL.

[38]  Ian E. Holliday,et al.  The coding of spatial position by the human visual system: Effects of spatial scale and contrast , 1992, Vision Research.

[39]  James R. Eagan,et al.  Low-level components of analytic activity in information visualization , 2005, IEEE Symposium on Information Visualization, 2005. INFOVIS 2005..

[40]  Michael Gleicher,et al.  Sequence Surveyor: Leveraging Overview for Scalable Genomic Alignment Visualization , 2011, IEEE Transactions on Visualization and Computer Graphics.

[41]  Jason M Haberman,et al.  Correspondences Rapid extraction of mean emotion and gender from sets of faces , 2007 .

[42]  M. Morgan,et al.  Efficiency of locating centres of dot-clusters by human observers , 1991, Vision Research.

[43]  Robert F. Hess,et al.  The coding of spatial position by the human visual system: Effects of spatial scale and retinal eccentricity , 1994, Vision Research.

[44]  Lisa A. Best,et al.  Perception of Linear and Nonlinear Trends: Using Slope and Curvature Information to Make Trend Discriminations , 2007, Perceptual and motor skills.

[45]  Ken Nakayama,et al.  Serial and parallel processing of visual feature conjunctions , 1986, Nature.

[46]  Jessica Lin,et al.  Visually mining and monitoring massive time series , 2004, KDD.

[47]  DAVID WHITAKER,et al.  Centroid Analysis Predicts Visual Localization of First- and Second-order Stimuli , 1996, Vision Research.

[48]  Matthew O. Ward,et al.  Measuring Data Abstraction Quality in Multiresolution Visualizations , 2006, IEEE Transactions on Visualization and Computer Graphics.

[49]  T. Callaghan Dimensional interaction of hue and brightness in preattentive field segregation , 1984, Perception & psychophysics.

[50]  Andrew B. Leber,et al.  Coordination of Voluntary and Stimulus-Driven Attentional Control in Human Cortex , 2005, Psychological science.

[51]  R. Watt,et al.  The computation of orientation statistics from visual texture , 1997, Vision Research.

[52]  Melanie Tory,et al.  Rethinking Visualization: A High-Level Taxonomy , 2004 .

[53]  James T. Enns,et al.  High-speed visual estimation using preattentive processing , 1996, TCHI.

[54]  Colin Ware,et al.  Designing a better weather display , 2013, Inf. Vis..

[55]  David Whitney,et al.  Reference repulsion in the categorical perception of biological motion , 2012, Vision Research.

[56]  Jean-Daniel Fekete,et al.  Task taxonomy for graph visualization , 2006, BELIV '06.

[57]  A. Treisman,et al.  Attentional spread in the statistical processing of visual displays , 2005, Perception & psychophysics.

[58]  Christopher Rao,et al.  Graphs in Statistical Analysis , 2010 .

[59]  James T. Enns,et al.  Attention and Visual Memory in Visualization and Computer Graphics , 2012, IEEE Transactions on Visualization and Computer Graphics.

[60]  Harold Pashler,et al.  Symmetry detection and visual attention: a “binary-map” hypothesis , 2002, Vision Research.

[61]  Petra Isenberg,et al.  Evaluation of alternative glyph designs for time series data in a small multiple setting , 2013, CHI.

[62]  Sarah A. Helseth,et al.  Flexible visual processing of spatial relationships , 2012, Cognition.

[63]  Ronald A. Rensink On the Prospects for a Science of Visualization , 2014, Handbook of Human Centric Visualization.

[64]  Mark A. Livingston,et al.  Evaluation of Trend Localization with Multi-Variate Visualizations , 2011, IEEE Transactions on Visualization and Computer Graphics.

[65]  Eileen Kowler,et al.  Shapes, surfaces and saccades , 1999, Vision Research.

[66]  I. Utochkin,et al.  Ensemble summary statistics as a basis for rapid visual categorization. , 2015, Journal of vision.

[67]  Bradley J. Morris,et al.  Comparing Data Sets: Implicit Summaries of the Statistical Properties of Number Sets , 2015, Cogn. Sci..

[68]  Robert E. Roth,et al.  Cartographic Interaction Primitives: Framework and Synthesis , 2012 .

[69]  William Prinzmetal,et al.  Color singleton pop-out does not always poop out: An alternative to visual search , 2006, Psychonomic bulletin & review.

[70]  J. D. Johnson,et al.  Attention and Memory , 1998 .

[71]  S. Franconeri The Nature and Status of Visual Resources , 2013 .

[72]  Jeffrey Heer,et al.  Sizing the horizon: the effects of chart size and layering on the graphical perception of time series visualizations , 2009, CHI.

[73]  C. Chubb,et al.  A 'dipper' function for texture discrimination based on orientation variance. , 2008, Journal of vision.

[74]  Tadasu Oyama,et al.  Perceptual Grouping as a Function of Proximity , 1961 .

[75]  G. Boynton,et al.  Global feature-based attention for motion and color , 2003, Vision Research.

[76]  Jeffrey Heer,et al.  Animated Transitions in Statistical Data Graphics , 2007, IEEE Transactions on Visualization and Computer Graphics.

[77]  J. Duncan,et al.  Visual search and stimulus similarity. , 1989, Psychological review.

[78]  Jason M Haberman,et al.  From Textures to Crowds : Multiple Levels of Summary Statistical Perception , 2017 .

[79]  Steven Franconeri,et al.  Perception of Average Value in Multiclass Scatterplots , 2013, IEEE Transactions on Visualization and Computer Graphics.

[80]  Ronald A. Rensink,et al.  The Perception of Correlation in Scatterplots , 2010, Comput. Graph. Forum.

[81]  Andrew Hollingworth,et al.  New objects do not capture attention without a sensory transient , 2010, Attention, perception & psychophysics.

[82]  D. Pelli,et al.  Crowding is unlike ordinary masking: distinguishing feature integration from detection. , 2004, Journal of vision.

[83]  Steven Franconeri,et al.  ISOTYPE Visualization: Working Memory, Performance, and Engagement with Pictographs , 2015, CHI.

[84]  Heidrun Schumann,et al.  A Design Space of Visualization Tasks , 2013, IEEE Transactions on Visualization and Computer Graphics.

[85]  Krzysztof Z. Gajos,et al.  Evaluation of Artery Visualizations for Heart Disease Diagnosis , 2011, IEEE Transactions on Visualization and Computer Graphics.

[86]  I. Rock,et al.  The legacy of Gestalt psychology. , 1990, Scientific American.

[87]  Andrew P. Duchon,et al.  The human visual system averages speed information , 1992, Vision Research.

[88]  Steven L Franconeri,et al.  Selecting and tracking multiple objects. , 2015, Wiley interdisciplinary reviews. Cognitive science.

[89]  L. Feigenson,et al.  Multiple Spatially Overlapping Sets Can Be Enumerated in Parallel , 2006, Psychological science.

[90]  Daniel J. Graham,et al.  Preference for art: similarity, statistics, and selling price , 2010, Electronic Imaging.

[91]  David Whitney,et al.  How Capacity Limits of Attention Influence Information Visualization Effectiveness , 2012, IEEE Transactions on Visualization and Computer Graphics.

[92]  John T. Stasko,et al.  The Information Mural: A Technique for Displaying and Navigating Large Information Spaces , 1998, IEEE Trans. Vis. Comput. Graph..

[93]  Jeffrey Heer,et al.  A tour through the visualization zoo , 2010, ACM Queue.

[94]  Steven L Franconeri,et al.  Average orientation is more accessible through object boundaries than surface features. , 2012, Journal of experimental psychology. Human perception and performance.

[95]  G. Alvarez Representing multiple objects as an ensemble enhances visual cognition , 2011, Trends in Cognitive Sciences.

[96]  Scott N. J. Watamaniuk,et al.  Direction Perception in Complex Dynamic Displays: the Integration of Dir~~tion Information , 1988 .

[97]  Diane M. Beck,et al.  Task-relevant and Task-irrelevant Dimensions Are Modulated Independently at a Task-irrelevant Location , 2012, Journal of Cognitive Neuroscience.

[98]  D. Burr,et al.  Vision senses number directly. , 2009, Journal of vision.

[99]  G. Fouriezos,et al.  Visual statistical decisions , 2008, Perception & psychophysics.

[100]  Michael Gleicher,et al.  Quantity estimation in visualizations of tagged text , 2013, CHI.

[101]  Michael Gleicher,et al.  Serendip: Topic model-driven visual exploration of text corpora , 2014, 2014 IEEE Conference on Visual Analytics Science and Technology (VAST).

[102]  Silvia Miksch,et al.  Bertin was Right: An Empirical Evaluation of Indexing to Compare Multivariate Time‐Series Data Using Line Plots , 2011, Comput. Graph. Forum.

[103]  Andreas Buja,et al.  Interactive High-Dimensional Data Visualization , 1996 .

[104]  David Melcher,et al.  Characterizing ensemble statistics: mean size is represented across multiple frames of reference , 2014, Attention, perception & psychophysics.

[105]  Michael Gleicher,et al.  Task-driven evaluation of aggregation in time series visualization , 2014, CHI.

[106]  Harold Pashler,et al.  A Boolean map theory of visual attention. , 2007, Psychological review.

[107]  K. Ottenbacher,et al.  The statistical analysis of single-subject data: a comparative examination. , 1994, Physical therapy.

[108]  A. Oliva,et al.  The Representation of Simple Ensemble Visual Features Outside the Focus of Attention , 2008, Psychological science.

[109]  James T. Enns,et al.  Clusters Precede Shapes in Perceptual Organization , 1997 .

[110]  David S. Ebert,et al.  Visualization and computer graphics , 2007 .

[111]  Steven L Franconeri,et al.  Common-Fate Grouping as Feature Selection , 2011, Psychological science.

[112]  Pierre Dragicevic,et al.  The Not-so-Staggering Effect of Staggered Animated Transitions on Visual Tracking , 2014, IEEE Transactions on Visualization and Computer Graphics.

[113]  S BoothKellogg,et al.  High-speed visual estimation using preattentive processing , 1996 .

[114]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.

[115]  S. Dakin Information limit on the spatial integration of local orientation signals. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[116]  Thomas W. Calvert,et al.  Moticons: : detection, distraction and task , 2003, Int. J. Hum. Comput. Stud..

[117]  Charles-Clemens Rüling Towards a conceptual framework , 2002 .

[118]  Tamara Munzner,et al.  A Multi-Level Typology of Abstract Visualization Tasks , 2013, IEEE Transactions on Visualization and Computer Graphics.

[119]  David J. Field,et al.  Contour integration by the human visual system: Evidence for a local “association field” , 1993, Vision Research.

[120]  T. Callaghan Interference and dominance in texture segregation: Hue, geometric form, and line orientation , 1989, Perception & psychophysics.

[121]  M. Webster,et al.  Perceiving the average hue of color arrays. , 2014, Journal of the Optical Society of America. A, Optics, image science, and vision.

[122]  A. Treisman,et al.  Statistical processing: computing the average size in perceptual groups , 2005, Vision Research.

[123]  Jason Dykes,et al.  Configuring Hierarchical Layouts to Address Research Questions , 2009, IEEE Transactions on Visualization and Computer Graphics.

[124]  Christopher D. Wickens,et al.  The Proximity Compatibility Principle: Its Psychological Foundation and Relevance to Display Design , 1995, Hum. Factors.

[125]  Ben Shneiderman,et al.  The eyes have it: a task by data type taxonomy for information visualizations , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.

[126]  Antonio Torralba,et al.  Building the gist of a scene: the role of global image features in recognition. , 2006, Progress in brain research.

[127]  Steven Franconeri,et al.  Comparing averages in time series data , 2012, CHI.

[128]  Heidrun Schumann,et al.  Towards a conceptual framework for visual analytics of time and time-oriented data , 2007, 2007 Winter Simulation Conference.

[129]  P. Cavanagh,et al.  Flexible cognitive resources: competitive content maps for attention and memory , 2013, Trends in Cognitive Sciences.