Quality Metrics for Information Visualization

The visualization community has developed to date many intuitions and understandings of how to judge the quality of views in visualizing data. The computation of a visualization's quality and usefulness ranges from measuring clutter and overlap, up to the existence and perception of specific (visual) patterns. This survey attempts to report, categorize and unify the diverse understandings and aims to establish a common vocabulary that will enable a wide audience to understand their differences and subtleties. For this purpose, we present a commonly applicable quality metric formalization that should detail and relate all constituting parts of a quality metric. We organize our corpus of reviewed research papers along the data types established in the information visualization community: multi‐ and high‐dimensional, relational, sequential, geospatial and text data. For each data type, we select the visualization subdomains in which quality metrics are an active research field and report their findings, reason on the underlying concepts, describe goals and outline the constraints and requirements. One central goal of this survey is to provide guidance on future research opportunities for the field and outline how different visualization communities could benefit from each other by applying or transferring knowledge to their respective subdomain. Additionally, we aim to motivate the visualization community to compare computed measures to the perception of humans.

[1]  Jean-Daniel Fekete,et al.  Magnostics: Image-Based Search of Interesting Matrix Views for Guided Network Exploration , 2017, IEEE Transactions on Visualization and Computer Graphics.

[2]  Georgia Albuquerque,et al.  Data Driven Color Mapping , 2011, EuroVA@EuroVis.

[3]  Daniel A. Keim,et al.  NEREx: Named‐Entity Relationship Exploration in Multi‐Party Conversations , 2017, Comput. Graph. Forum.

[4]  Luigi Di Caro,et al.  Analyzing the Role of Dimension Arrangement for Data Visualization in Radviz , 2010, PAKDD.

[5]  Stefan Niermann Optimizing the Ordering of Tables With Evolutionary Computation , 2005 .

[6]  Robert S. Laramee,et al.  Cartographic Treemaps for Visualization of Public Healthcare Data , 2017, CGVC.

[7]  J. Rodgers,et al.  Thirteen ways to look at the correlation coefficient , 1988 .

[8]  Jeffrey Heer,et al.  Multi-Scale Banking to 45 Degrees , 2006, IEEE Transactions on Visualization and Computer Graphics.

[9]  Jordi Petit,et al.  Experiments on the minimum linear arrangement problem , 2003, ACM J. Exp. Algorithmics.

[10]  David Harel,et al.  A Multi-scale Algorithm for the Linear Arrangement Problem , 2002, WG.

[11]  Paul J. Schweitzer,et al.  Problem Decomposition and Data Reorganization by a Clustering Technique , 1972, Oper. Res..

[12]  Kanit Wongsuphasawat,et al.  Voyager: Exploratory Analysis via Faceted Browsing of Visualization Recommendations , 2016, IEEE Transactions on Visualization and Computer Graphics.

[13]  H. P. Friedman,et al.  The surgical implications of physiologic patterns in myocardial infarction shock. , 1972, Surgery.

[14]  Martin Wattenberg,et al.  Stacked Graphs – Geometry & Aesthetics , 2008, IEEE Transactions on Visualization and Computer Graphics.

[15]  Mao Lin Huang,et al.  Two axes re-ordering methods in parallel coordinates plots , 2016, J. Vis. Lang. Comput..

[16]  Ben Shneiderman,et al.  Tree visualization with tree-maps: 2-d space-filling approach , 1992, TOGS.

[17]  Christian Rohrdantz,et al.  Exploratory Text Analysis using Lexical Episode Plots , 2015, EuroVis.

[18]  Cynthia A. Brewer,et al.  Mapping Mortality: Evaluating Color Schemes for Choropleth Maps , 1997 .

[19]  Ben Shneiderman,et al.  Readability metric feedback for aiding node-link visualization designers , 2015, IBM J. Res. Dev..

[20]  Camilla Forsell,et al.  Evaluation of Parallel Coordinates: Overview, Categorization and Guidelines for Future Research , 2016, IEEE Transactions on Visualization and Computer Graphics.

[21]  Jing Li,et al.  Judging Correlation from Scatterplots and Parallel Coordinate Plots , 2010, Inf. Vis..

[22]  Leland Wilkinson,et al.  Transforming Scagnostics to Reveal Hidden Features , 2014, IEEE Transactions on Visualization and Computer Graphics.

[23]  Ricardo Baeza-Yates,et al.  Good fonts for dyslexia , 2013, ASSETS.

[24]  Heidrun Schumann,et al.  Visualizing time-oriented data - A systematic view , 2007, Comput. Graph..

[25]  John W. Tukey,et al.  A Projection Pursuit Algorithm for Exploratory Data Analysis , 1974, IEEE Transactions on Computers.

[26]  Michael Gleicher,et al.  Scatterplots: Tasks, Data, and Designs , 2018, IEEE Transactions on Visualization and Computer Graphics.

[27]  Leland Wilkinson,et al.  ScagExplorer: Exploring Scatterplots by Their Scagnostics , 2014, 2014 IEEE Pacific Visualization Symposium.

[28]  Alexandru Telea,et al.  Skeleton-Based Scagnostics , 2018, IEEE Transactions on Visualization and Computer Graphics.

[29]  Ben Shneiderman,et al.  Readings in information visualization - using vision to think , 1999 .

[30]  Camilla Forsell,et al.  Task-based evaluation of multirelational 3D and standard 2D parallel coordinates , 2007, Electronic Imaging.

[31]  Daniel Weiskopf,et al.  State of the Art of Parallel Coordinates , 2013, Eurographics.

[32]  Lifeng Zhu,et al.  Line Graph or Scatter Plot? Automatic Selection of Methods for Visualizing Trends in Time Series , 2018, IEEE Transactions on Visualization and Computer Graphics.

[33]  Weiwei Cui,et al.  Overview of Text Visualization Techniques , 2016 .

[34]  James T. Enns,et al.  Attention and Visual Memory in Visualization and Computer Graphics , 2012, IEEE Transactions on Visualization and Computer Graphics.

[35]  Martin Wattenberg,et al.  Participatory Visualization with Wordle , 2009, IEEE Transactions on Visualization and Computer Graphics.

[36]  Alan J. Dix,et al.  Enabling Automatic Clutter Reduction in Parallel Coordinate Plots , 2006, IEEE Transactions on Visualization and Computer Graphics.

[37]  Katharina Reinecke,et al.  Infographic Aesthetics: Designing for the First Impression , 2015, CHI.

[38]  Marcus A. Magnor,et al.  Selecting Coherent and Relevant Plots in Large Scatterplot Matrices , 2012, Comput. Graph. Forum.

[39]  Chun-Houh Chen GENERALIZED ASSOCIATION PLOTS: INFORMATION VISUALIZATION VIA ITERATIVELY GENERATED CORRELATION MATRICES , 2002 .

[40]  Georges G. Grinstein,et al.  Vectorized Radviz and Its Application to Multiple Cluster Datasets , 2008, IEEE Transactions on Visualization and Computer Graphics.

[41]  U. Brandes,et al.  Explorations into the Visualization of Policy Networks , 1999 .

[42]  Jos B. T. M. Roerdink,et al.  Visualizing High‐Dimensional Structures by Dimension Ordering and Filtering using Subspace Analysis , 2011, Comput. Graph. Forum.

[43]  Maria Cristina Ferreira de Oliveira,et al.  Seeing beyond reading: a survey on visual text analytics , 2012, WIREs Data Mining Knowl. Discov..

[44]  Chi-Wing Fu,et al.  Is There a Robust Technique for Selecting Aspect Ratios in Line Charts? , 2018, IEEE Transactions on Visualization and Computer Graphics.

[45]  Matthew D. Cooper,et al.  A Screen Space Quality Method for Data Abstraction , 2008, Comput. Graph. Forum.

[46]  Michael Dixon,et al.  Google Earth Engine: Planetary-scale geospatial analysis for everyone , 2017 .

[47]  Joseph H. Goldberg,et al.  Enterprise Network Monitoring Using Treemaps , 2005 .

[48]  Antti Oulasvirta,et al.  Towards Perceptual Optimization of the Visual Design of Scatterplots , 2017, IEEE Transactions on Visualization and Computer Graphics.

[49]  Anastasia Bezerianos,et al.  A Systematic Review of Experimental Studies on Data Glyphs , 2017, IEEE Transactions on Visualization and Computer Graphics.

[50]  Heidrun Schumann,et al.  Axes-based visualizations with radial layouts , 2004, SAC '04.

[51]  Robert Kosara,et al.  Pargnostics: Screen-Space Metrics for Parallel Coordinates , 2010, IEEE Transactions on Visualization and Computer Graphics.

[52]  Jian Pei,et al.  Online Visual Analytics of Text Streams , 2015, IEEE Transactions on Visualization and Computer Graphics.

[53]  Tobias Schreck,et al.  Visual Exploration of Large Scatter Plot Matrices by Pattern Recommendation based on Eye Tracking , 2017, ESIDA@IUI.

[54]  Martin Halvey,et al.  An assessment of tag presentation techniques , 2007, WWW '07.

[55]  Stefan Berchtold,et al.  Similarity clustering of dimensions for an enhanced visualization of multidimensional data , 1998, Proceedings IEEE Symposium on Information Visualization (Cat. No.98TB100258).

[56]  G. Santucci,et al.  SpringView: cooperation of radviz and parallel coordinates for view optimization and clutter reduction , 2005, Coordinated and Multiple Views in Exploratory Visualization (CMV'05).

[57]  Tamara Munzner,et al.  A Taxonomy of Visual Cluster Separation Factors , 2012, Comput. Graph. Forum.

[58]  Niklas Elmqvist,et al.  Graphical Perception of Multiple Time Series , 2010, IEEE Transactions on Visualization and Computer Graphics.

[59]  Min Chen,et al.  An Information-theoretic Framework for Visualization , 2010, IEEE Transactions on Visualization and Computer Graphics.

[60]  Yusef Hassan-Montero,et al.  Improving Tag-Clouds as Visual Information Retrieval Interfaces , 2024, 2401.04947.

[61]  Daniela Karin Rosner,et al.  Tag Clouds: Data Analysis Tool or Social Signaller? , 2008, Proceedings of the 41st Annual Hawaii International Conference on System Sciences (HICSS 2008).

[62]  Marcus A. Magnor,et al.  Quality-Based Visualization Matrices , 2009, VMV.

[63]  E. Tufte Beautiful Evidence , 2006 .

[64]  Alex Endert,et al.  Data and Task Based Effectiveness of Basic Visualizations , 2017, ArXiv.

[65]  Tamara Munzner,et al.  TimeLineCurator: Interactive Authoring of Visual Timelines from Unstructured Text , 2016, IEEE Transactions on Visualization and Computer Graphics.

[66]  Thomas Ertl,et al.  VarifocalReader — In-Depth Visual Analysis of Large Text Documents , 2014, IEEE Transactions on Visualization and Computer Graphics.

[67]  Wolfgang Kienreich,et al.  On the Beauty and Usability of Tag Clouds , 2008, 2008 12th International Conference Information Visualisation.

[68]  Daniel A. Keim,et al.  Guided Sketching for Visual Search and Exploration in Large Scatter Plot Spaces , 2014, EuroVA@EuroVis.

[69]  Stephen G. Kobourov,et al.  The State of the Art in Cartograms , 2016, Comput. Graph. Forum.

[70]  Ulrik Brandes,et al.  Generative Data Models for Validation and Evaluation of Visualization Techniques , 2016, BELIV '16.

[71]  Shimei Pan,et al.  Interactive, topic-based visual text summarization and analysis , 2009, CIKM.

[72]  John T. Stasko,et al.  TypoTweet Maps: Characterizing Urban Areas through Typographic Social Media Visualization , 2017, EuroVis.

[73]  Martin Wattenberg,et al.  TIMELINESTag clouds and the case for vernacular visualization , 2008, INTR.

[74]  Jean-Daniel Fekete,et al.  An extended evaluation of the readability of tapered, animated, and textured directed-edge representations in node-link graphs , 2011, 2011 IEEE Pacific Visualization Symposium.

[75]  W. Cleveland,et al.  Judgments of Circle Sizes on Statistical Maps , 1982 .

[76]  Alberto Sánchez,et al.  A comparative study between RadViz and Star Coordinates , 2016, IEEE Transactions on Visualization and Computer Graphics.

[77]  Michael Burch,et al.  Evaluating Partially Drawn Links for Directed Graph Edges , 2011, GD.

[78]  Paul J. Schweitzer,et al.  Identification of Data Structures and Relationships by Matrix Reordering Techniques. , 1969 .

[79]  Nancy Argüelles,et al.  Author ' s , 2008 .

[80]  Min Chen,et al.  Glyph-based Visualization: Foundations, Design Guidelines, Techniques and Applications , 2013, Eurographics.

[81]  Kevin Larson,et al.  How does typeface familiarity affect reading performance and reader preference , 2013 .

[82]  Paul Whitney,et al.  The need for metrics in visual information analysis , 1997, NPIV '97.

[83]  Daniel A. Keim,et al.  Visual quality metrics and human perception: an initial study on 2D projections of large multidimensional data , 2010, AVI.

[84]  Daniel A. Keim,et al.  Feature Alignment for the Analysis of Verbatim Text Transcripts , 2017, EuroVA@EuroVis.

[85]  W. S. Robinson A Method for Chronologically Ordering Archaeological Deposits , 1951, American Antiquity.

[86]  Daniel A. Keim,et al.  Pixnostics: Towards Measuring the Value of Visualization , 2006, 2006 IEEE Symposium On Visual Analytics Science And Technology.

[87]  Jean-Daniel Fekete,et al.  Task taxonomy for graph visualization , 2006, BELIV '06.

[88]  T. J. Jankun-Kelly,et al.  MoireGraphs: radial focus+context visualization and interaction for graphs with visual nodes , 2003, IEEE Symposium on Information Visualization 2003 (IEEE Cat. No.03TH8714).

[89]  John P. Snyder,et al.  Map projections used by the U.S. Geological Survey , 1982 .

[90]  Alan J. Dix,et al.  A Taxonomy of Clutter Reduction for Information Visualisation , 2007, IEEE Transactions on Visualization and Computer Graphics.

[91]  Ebad Banissi,et al.  The Design Space of Typeface , 2014 .

[92]  William S. Cleveland,et al.  A Model for Studying Display Methods of Statistical Graphics , 1993 .

[93]  Daniel A. Keim,et al.  What you see is what you can change: Human-centered machine learning by interactive visualization , 2017, Neurocomputing.

[94]  Giuseppe Santucci,et al.  Improving 2D scatterplots effectiveness through sampling, displacement, and user perception , 2005, Ninth International Conference on Information Visualisation (IV'05).

[95]  Ingwer Borg,et al.  Performance of Snow Flakes, Suns, and Factorial Suns in the Graphical Representation of Multivariate Data , 1992 .

[96]  JOSEP DÍAZ,et al.  A survey of graph layout problems , 2002, CSUR.

[97]  Mats Lind,et al.  Many-to-Many Relational Parallel Coordinates Displays , 2009, 2009 13th International Conference Information Visualisation.

[98]  Alexander Wolff,et al.  Selecting the Aspect Ratio of a Scatter Plot Based on Its Delaunay Triangulation , 2013, IEEE Transactions on Visualization and Computer Graphics.

[99]  Michael Cardew-Hall,et al.  The folksonomy tag cloud: when is it useful? , 2008, J. Inf. Sci..

[100]  Richard Bellman,et al.  Adaptive Control Processes: A Guided Tour , 1961, The Mathematical Gazette.

[101]  Kwan-Liu Ma,et al.  Semantic‐Preserving Word Clouds by Seam Carving , 2011, Comput. Graph. Forum.

[102]  Sofie Beier,et al.  Typeface Legibility: Towards defining familiarity , 2009 .

[103]  Jeffrey Heer,et al.  Surprise! Bayesian Weighting for De-Biasing Thematic Maps , 2017, IEEE Transactions on Visualization and Computer Graphics.

[104]  Anne E. Trefethen,et al.  Rule‐based Visual Mappings – with a Case Study on Poetry Visualization , 2013, Comput. Graph. Forum.

[105]  Jarke J. van Wijk,et al.  Flexible Linked Axes for Multivariate Data Visualization , 2011, IEEE Transactions on Visualization and Computer Graphics.

[106]  E. Anderson A SEMIGRAPHICAL METHOD FOR THE ANALYSIS OF COMPLEX PROBLEMS. , 1957, Proceedings of the National Academy of Sciences of the United States of America.

[107]  Bettina Speckmann,et al.  Geo word clouds , 2016, 2016 IEEE Pacific Visualization Symposium (PacificVis).

[108]  Kanit Wongsuphasawat,et al.  Voyager 2: Augmenting Visual Analysis with Partial View Specifications , 2017, CHI.

[109]  Min Chen,et al.  Visual Multiplexing , 2014, Comput. Graph. Forum.

[110]  Jaroslav Nesetril,et al.  Towards an Aesthetic Invariant for Graph Drawing , 2001, GD.

[111]  Andrew Lumsdaine,et al.  A comparison of vertex ordering algorithms for large graph visualization , 2007, 2007 6th International Asia-Pacific Symposium on Visualization.

[112]  Jacques Bertin Théorie matricielle de la graphique , 1981 .

[113]  Daniel A. Keim,et al.  Rolled‐out Wordles: A Heuristic Method for Overlap Removal of 2D Data Representatives , 2012, Comput. Graph. Forum.

[114]  Ilaria Liccardi,et al.  An Empirical Study on the Reliability of Perceiving Correlation Indices using Scatterplots , 2017, Comput. Graph. Forum.

[115]  Daniel A. Keim,et al.  PixelMaps: a new visual data mining approach for analyzing large spatial data sets , 2003, Third IEEE International Conference on Data Mining.

[116]  John T. Stasko,et al.  Visualizing Social Media Content with SentenTree , 2017, IEEE Transactions on Visualization and Computer Graphics.

[117]  Michaël Aupetit,et al.  SepMe: 2002 New visual separation measures , 2016, 2016 IEEE Pacific Visualization Symposium (PacificVis).

[118]  Daniel A. Keim,et al.  A Spectral Visualization System for Analyzing Financial Time Series Data , 2006, EuroVis.

[119]  Daniel A. Keim,et al.  ColorCAT: Guided Design of Colormaps for Combined Analysis Tasks , 2015, EuroVis.

[120]  Stephen G. Kobourov,et al.  Quantitative Measures for Cartogram Generation Techniques , 2015, Comput. Graph. Forum.

[121]  Steffen Lohmann,et al.  Comparison of Tag Cloud Layouts: Task-Related Performance and Visual Exploration , 2009, INTERACT.

[122]  Georges G. Grinstein,et al.  Iconographic Displays For Visualizing Multidimensional Data , 1988, Proceedings of the 1988 IEEE International Conference on Systems, Man, and Cybernetics.

[123]  Alfred Inselberg,et al.  The plane with parallel coordinates , 1985, The Visual Computer.

[124]  Matthew O. Ward,et al.  Interactive Data Visualization - Foundations, Techniques, and Applications , 2010 .

[125]  Roberto Tamassia,et al.  Handbook on Graph Drawing and Visualization , 2013 .

[126]  Michael J. Muller,et al.  Getting our head in the clouds: toward evaluation studies of tagclouds , 2007, CHI.

[127]  Carl Gutwin,et al.  Seeing things in the clouds: the effect of visual features on tag cloud selections , 2008, Hypertext.

[128]  William Ribarsky,et al.  VAiRoma: A Visual Analytics System for Making Sense of Places, Times, and Events in Roman History , 2016, IEEE Transactions on Visualization and Computer Graphics.

[129]  Camilla Forsell,et al.  On the usability of three-dimensional display in parallel coordinates: Evaluating the efficiency of identifying two-dimensional relationships , 2014, Inf. Vis..

[130]  G. Andrienko,et al.  Exploratory analysis of spatial and temporal data , 2013 .

[131]  C. Mueller,et al.  Sparse Matrix Reordering Algorithms for Cluster Identification , 2004 .

[132]  Hanspeter Pfister,et al.  Beyond Memorability: Visualization Recognition and Recall , 2016, IEEE Transactions on Visualization and Computer Graphics.

[133]  Azriel Rosenfeld,et al.  Sequential Operations in Digital Picture Processing , 1966, JACM.

[134]  K. R. Priya Grounded Theory Methodology , 2013 .

[135]  Greta Franzini,et al.  On Close and Distant Reading in Digital Humanities: A Survey and Future Challenges , 2015, EuroVis.

[136]  Leland Wilkinson,et al.  Scagnostics Distributions , 2008 .

[137]  Heidrun Schumann,et al.  Visual support for the understanding of simulation processes , 2009, 2009 IEEE Pacific Visualization Symposium.

[138]  Matthew O. Ward,et al.  Visual Exploration of Time‐Series Data with Shape Space Projections , 2011, Comput. Graph. Forum.

[139]  Michael J. Haass,et al.  Data Visualization Saliency Model: A Tool for Evaluating Abstract Data Visualizations , 2018, IEEE Transactions on Visualization and Computer Graphics.

[140]  Hanspeter Pfister,et al.  Guidelines for Effective Usage of Text Highlighting Techniques , 2016, IEEE Transactions on Visualization and Computer Graphics.

[141]  Andrew Lumsdaine,et al.  Interpreting large visual similarity matrices , 2007, 2007 6th International Asia-Pacific Symposium on Visualization.

[142]  Xiaohui Yu,et al.  A Perception-Driven Approach to Supervised Dimensionality Reduction for Visualization , 2018, IEEE Transactions on Visualization and Computer Graphics.

[143]  Helen C. Purchase,et al.  Which Aesthetic has the Greatest Effect on Human Understanding? , 1997, GD.

[144]  Ebad Banissi,et al.  Typographic Sets: Labelled Set Elements with Font Attributes , 2016, SetVR@Diagrams.

[145]  Chi-Wing Fu,et al.  Revisiting Stress Majorization as a Unified Framework for Interactive Constrained Graph Visualization , 2018, IEEE Transactions on Visualization and Computer Graphics.

[146]  Steven Franconeri,et al.  Perceptual Biases in Font Size as a Data Encoding , 2018, IEEE Transactions on Visualization and Computer Graphics.

[147]  Pat Hanrahan,et al.  Visualizing route maps , 2001 .

[148]  John P. Snyder,et al.  Map Projections Used by the US Geological Survey: Geological Survey Bulletin No. 1532 , 1984 .

[149]  M. Monmonier How to Lie with Maps , 1991 .

[150]  Daniel A. Keim,et al.  Literature Fingerprinting: A New Method for Visual Literary Analysis , 2007, 2007 IEEE Symposium on Visual Analytics Science and Technology.

[151]  Michael Burch,et al.  Comparative Eye Tracking Study on Node-Link Visualizations of Trajectories , 2014, IEEE Transactions on Visualization and Computer Graphics.

[152]  Eser Kandogan Star Coordinates: A Multi-dimensional Visualization Technique with Uniform Treatment of Dimensions , 2000 .

[153]  Martin Wattenberg A note on space-filling visualizations and space-filling curves , 2005, IEEE Symposium on Information Visualization, 2005. INFOVIS 2005..

[154]  Padhraic Smyth,et al.  From Data Mining to Knowledge Discovery in Databases , 1996, AI Mag..

[155]  Alan J. Dix,et al.  The plot, the clutter, the sampling and its lens: occlusion measures for automatic clutter reduction , 2006, AVI '06.

[156]  Hans-Peter Seidel,et al.  An Edge-Bundling Layout for Interactive Parallel Coordinates , 2014, 2014 IEEE Pacific Visualization Symposium.

[157]  Holger Theisel,et al.  A study on quality metrics vs. human perception: Can visual measures help us to filter visualizations of interest? , 2015, it Inf. Technol..

[158]  Robert L. Harris,et al.  Information Graphics: A Comprehensive Illustrated Reference , 1996 .

[159]  Gilles Caraux,et al.  PermutMatrix: a graphical environment to arrange gene expression profiles in optimal linear order , 2005, Bioinform..

[160]  Andy Cockburn,et al.  Enhanced Spatial Stability with Hilbert and Moore Treemaps , 2013, IEEE Transactions on Visualization and Computer Graphics.

[161]  Daniel Weiskopf,et al.  Comparative eye-tracking evaluation of scatterplots and parallel coordinates , 2017, Vis. Informatics.

[162]  Issei Fujishiro,et al.  The elements of graphing data , 2005, The Visual Computer.

[163]  Ben Shneiderman,et al.  The eyes have it: a task by data type taxonomy for information visualizations , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.

[164]  Leland Wilkinson,et al.  The History of the Cluster Heat Map , 2009 .

[165]  Innar Liiv,et al.  Seriation and matrix reordering methods: An historical overview , 2010, Stat. Anal. Data Min..

[166]  Johannes Fuchs,et al.  ClockMap: Enhancing Circular Treemaps with Temporal Glyphs for Time-Series Data , 2012, EuroVis.

[167]  Ivan Viola,et al.  Exploring visual attention and saliency modeling for task-based visual analysis , 2018, Comput. Graph..

[168]  Daniel A. Keim,et al.  Guiding the Exploration of Scatter Plot Data Using Motif-Based Interest Measures , 2015, 2015 Big Data Visual Analytics (BDVA).

[169]  Leland Wilkinson,et al.  TimeSeer: Scagnostics for High-Dimensional Time Series , 2013, IEEE Transactions on Visualization and Computer Graphics.

[170]  Alan J. Dix,et al.  The sampling lens: making sense of saturated visualisations , 2005, CHI Extended Abstracts.

[171]  Anshul Vikram Pandey,et al.  Towards Understanding Human Similarity Perception in the Analysis of Large Sets of Scatter Plots , 2016, CHI.

[172]  Daniel A. Keim,et al.  Efficient Contrast Effect Compensation with Personalized Perception Models , 2015, Comput. Graph. Forum.

[173]  Herman Chernoff,et al.  The Use of Faces to Represent Points in k- Dimensional Space Graphically , 1973 .

[174]  Daniel A. Keim,et al.  ConToVi: Multi‐Party Conversation Exploration using Topic‐Space Views , 2016, Comput. Graph. Forum.

[175]  Andreas Kerren,et al.  Text visualization techniques: Taxonomy, visual survey, and community insights , 2015, 2015 IEEE Pacific Visualization Symposium (PacificVis).

[176]  W. Cleveland,et al.  Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods , 1984 .

[177]  Catherine B. Hurley,et al.  Pairwise Display of High-Dimensional Information via Eulerian Tours and Hamiltonian Decompositions , 2010 .

[178]  Georges G. Grinstein,et al.  DNA visual and analytic data mining , 1997, Proceedings. Visualization '97 (Cat. No. 97CB36155).

[179]  Eric Horvitz,et al.  Principles of mixed-initiative user interfaces , 1999, CHI '99.

[180]  Peter Eades,et al.  On the faithfulness of graph visualizations , 2012, 2013 IEEE Pacific Visualization Symposium (PacificVis).

[181]  Daniel A. Keim,et al.  A survey and task-based quality assessment of static 2D colormaps , 2015, Electronic Imaging.

[182]  Jonathan C. Roberts,et al.  Force-Directed Parallel Coordinates , 2013, 2013 17th International Conference on Information Visualisation.

[183]  Daniel Weiskopf,et al.  Bubble Treemaps for Uncertainty Visualization , 2018, IEEE Transactions on Visualization and Computer Graphics.

[184]  Yuanzhen Li,et al.  Feature congestion: a measure of display clutter , 2005, CHI.

[185]  Ebad Banissi,et al.  Evaluating lossiness and fidelity in information visualization , 2015, Electronic Imaging.

[186]  Rui Li,et al.  Colour-Enhanced Star Plot Glyphs: Can Salient Shape Characteristics Be Overcome? , 2009, Cartogr. Int. J. Geogr. Inf. Geovisualization.

[187]  Peter L. Brooks,et al.  Visualizing data , 1997 .

[188]  Ulrik Brandes,et al.  Social Networks , 2013, Handbook of Graph Drawing and Visualization.

[189]  Rolf Niedermeier,et al.  Towards optimal locality in mesh-indexings , 1997, Discret. Appl. Math..

[190]  Giuseppe Santucci,et al.  Quality Metrics for 2D Scatterplot Graphics: Automatically Reducing Visual Clutter , 2004, Smart Graphics.

[191]  Matthew O. Ward,et al.  Measuring Data Abstraction Quality in Multiresolution Visualizations , 2006, IEEE Transactions on Visualization and Computer Graphics.

[192]  D. Hilbert Ueber die stetige Abbildung einer Line auf ein Flächenstück , 1891 .

[193]  Richard F. Riesenfeld,et al.  A Survey of Radial Methods for Information Visualization , 2009, IEEE Transactions on Visualization and Computer Graphics.

[194]  Tamara Munzner,et al.  A Multi-Level Typology of Abstract Visualization Tasks , 2013, IEEE Transactions on Visualization and Computer Graphics.

[195]  Michael Hahsler,et al.  Getting Things in Order: An Introduction to the R Package seriation , 2008 .

[196]  Gennady L. Andrienko,et al.  Interactive visual tools to explore spatio-temporal variation , 2004, AVI.

[197]  Jarke J. van Wijk,et al.  A user study on visualizing directed edges in graphs , 2009, CHI.

[198]  Lawrence Hubert,et al.  SOME APPLICATIONS OF GRAPH THEORY AND RELATED NON‐METRIC TECHNIQUES TO PROBLEMS OF APPROXIMATE SERIATION: THE CASE OF SYMMETRIC PROXIMITY MEASURES , 1974 .

[199]  Alok N. Choudhary,et al.  Adaptive Grids for Clustering Massive Data Sets , 2001, SDM.

[200]  Jian Zhang,et al.  Mathematical foundations of arc length-based aspect ratio selection , 2016, 2016 IEEE Pacific Visualization Symposium (PacificVis).

[201]  Blaz Zupan,et al.  Data and text mining Visualization-based cancer microarray data classification analysis , 2007 .

[202]  Danielle Albers Szafir,et al.  Visualization-Aware Color Design , 2016, EuroVis.

[203]  Jacques Bertin,et al.  Semiologie graphique : les diagrammes les réseaux, les cartes , 1969 .

[204]  John P. Lewis,et al.  Eurographics/ Ieee-vgtc Symposium on Visualization 2009 Selecting Good Views of High-dimensional Data Using Class Consistency , 2022 .

[205]  Chris Weaver,et al.  Star Plots: How Shape Characteristics Influence Classification Tasks , 2009 .

[206]  Chun-Houh Chen,et al.  Methods for simultaneously identifying coherent local clusters with smooth global patterns in gene expression profiles , 2008, BMC Bioinformatics.

[207]  Pat Hanrahan,et al.  Show Me: Automatic Presentation for Visual Analysis , 2007, IEEE Transactions on Visualization and Computer Graphics.

[208]  Philippe Castagliola,et al.  A Comparison of the Readability of Graphs Using Node-Link and Matrix-Based Representations , 2004, IEEE Symposium on Information Visualization.

[209]  Alex Endert,et al.  Beyond Usability and Performance: A Review of User Experience-focused Evaluations in Visualization , 2016, BELIV '16.

[210]  Jarke J. van Wijk,et al.  Eurographics/ieee-vgtc Symposium on Visualization 2010 Evaluation of Cluster Identification Performance for Different Pcp Variants , 2022 .

[211]  Ben Shneiderman,et al.  LifeLines: visualizing personal histories , 1996, CHI.

[212]  Jaakko Peltonen,et al.  Parallel Coordinate Plots for Neighbor Retrieval , 2017, VISIGRAPP.

[213]  Jim Felici The Complete Manual of Typography: A Guide to Setting Perfect Type , 2002 .

[214]  Ben Shneiderman,et al.  Ordered and quantum treemaps: Making effective use of 2D space to display hierarchies , 2002, TOGS.

[215]  Marcus A. Magnor,et al.  Improving the visual analysis of high-dimensional datasets using quality measures , 2010, 2010 IEEE Symposium on Visual Analytics Science and Technology.

[216]  Fatih Korkmaz,et al.  Feedback-driven interactive exploration of large multidimensional data supported by visual classifier , 2014, 2014 IEEE Conference on Visual Analytics Science and Technology (VAST).

[217]  Waldo R. Tobler Thirty Five Years of Computer Cartograms , 2004 .

[218]  Richard Brath,et al.  Metrics for effective information visualization , 1997, Proceedings of VIZ '97: Visualization Conference, Information Visualization Symposium and Parallel Rendering Symposium.

[219]  S. Elo,et al.  Qualitative Content Analysis , 2014 .

[220]  David H. Laidlaw,et al.  Colorgorical: Creating discriminable and preferable color palettes for information visualization , 2017, IEEE Transactions on Visualization and Computer Graphics.

[221]  Daniel A. Keim,et al.  Visual data mining in large geospatial point sets , 2004, IEEE Computer Graphics and Applications.

[222]  Ioannis G. Tollis,et al.  Algorithms for Drawing Graphs: an Annotated Bibliography , 1988, Comput. Geom..

[223]  Mats Lind,et al.  Perceiving Patterns in Parallel Coordinates: Determining Thresholds for Identification of Relationships , 2008, Inf. Vis..

[224]  Jeffrey Heer,et al.  Perceptual Guidelines for Creating Rectangular Treemaps , 2010, IEEE Transactions on Visualization and Computer Graphics.

[225]  Jean-Daniel Fekete,et al.  Matrix Reordering Methods for Table and Network Visualization , 2016, Comput. Graph. Forum.

[226]  Matthew O. Ward,et al.  Clutter Reduction in Multi-Dimensional Data Visualization Using Dimension Reordering , 2004, IEEE Symposium on Information Visualization.

[227]  Anastasia Bezerianos,et al.  The Influence of Contour on Similarity Perception of Star Glyphs , 2014, IEEE Transactions on Visualization and Computer Graphics.

[228]  Lucy T. Nowell,et al.  ThemeRiver: Visualizing Thematic Changes in Large Document Collections , 2002, IEEE Trans. Vis. Comput. Graph..

[229]  Matthew O. Ward,et al.  Interactive hierarchical dimension ordering, spacing and filtering for exploration of high dimensional datasets , 2003, IEEE Symposium on Information Visualization 2003 (IEEE Cat. No.03TH8714).

[230]  Daniel A. Keim,et al.  Visual Analytics: Definition, Process, and Challenges , 2008, Information Visualization.

[231]  Colin Ware,et al.  Cognitive Measurements of Graph Aesthetics , 2002, Inf. Vis..

[232]  Marcus A. Magnor,et al.  Combining automated analysis and visualization techniques for effective exploration of high-dimensional data , 2009, 2009 IEEE Symposium on Visual Analytics Science and Technology.

[233]  Jeffery. M. Zacks,et al.  Bars and lines: A study of graphic communication , 1999, Memory & cognition.

[234]  Jonathan Goldstein,et al.  When Is ''Nearest Neighbor'' Meaningful? , 1999, ICDT.

[235]  Xiaoru Yuan,et al.  Dimension Projection Matrix/Tree: Interactive Subspace Visual Exploration and Analysis of High Dimensional Data , 2013, IEEE Transactions on Visualization and Computer Graphics.

[236]  Daniel Weiskopf,et al.  Evaluation of a Bundling Technique for Parallel Coordinates , 2011, GRAPP/IVAPP.

[237]  William S. Cleveland,et al.  The Shape Parameter of a Two-Variable Graph , 1988 .

[238]  Manfred Tscheligi,et al.  Semantically structured tag clouds: an empirical evaluation of clustered presentation approaches , 2009, CHI.

[239]  Daniel A. Keim,et al.  Designing Pixel-Oriented Visualization Techniques: Theory and Applications , 2000, IEEE Trans. Vis. Comput. Graph..

[240]  Michaël Aupetit,et al.  Data‐driven Evaluation of Visual Quality Measures , 2015, Comput. Graph. Forum.

[241]  Enrico Bertini,et al.  Quality Metrics in High-Dimensional Data Visualization: An Overview and Systematization , 2011, IEEE Transactions on Visualization and Computer Graphics.

[242]  Daniel W. Archambault,et al.  Can animation support the visualisation of dynamic graphs? , 2016, Inf. Sci..

[243]  M. Balzer,et al.  Voronoi treemaps , 2005, IEEE Symposium on Information Visualization, 2005. INFOVIS 2005..

[244]  Tamara Munzner,et al.  Dimensionality reduction for documents with nearest neighbor queries , 2015, Neurocomputing.

[245]  Ulrik Brandes,et al.  Untangling the Hairballs of Multi-Centered, Small-World Online Social Media Networks , 2015, J. Graph Algorithms Appl..

[246]  Jock D. Mackinlay,et al.  Automating the design of graphical presentations of relational information , 1986, TOGS.

[247]  Jeffrey Heer,et al.  Crowdsourcing graphical perception: using mechanical turk to assess visualization design , 2010, CHI.

[248]  Daniel A. Keim,et al.  Pixel based visual data mining of geo-spatial data , 2004, Comput. Graph..

[249]  Min Chen,et al.  Eurographics/ Ieee-vgtc Symposium on Visualization 2010 a Salience-based Quality Metric for Visualization , 2022 .

[250]  Oliver Deussen,et al.  EdWordle: Consistency-Preserving Word Cloud Editing , 2018, IEEE Transactions on Visualization and Computer Graphics.

[251]  Bettina Speckmann,et al.  Stable Treemaps via Local Moves , 2018, IEEE Transactions on Visualization and Computer Graphics.

[252]  Petra Isenberg,et al.  Evaluation of alternative glyph designs for time series data in a small multiple setting , 2013, CHI.

[253]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[254]  Rosane Minghim,et al.  Semantic Wordification of Document Collections , 2012, Comput. Graph. Forum.

[255]  Peter Eades,et al.  Proxy Graph: Visual Quality Metrics of Big Graph Sampling , 2017, IEEE Transactions on Visualization and Computer Graphics.

[256]  Olga Stepánková,et al.  Visualization of trends using RadViz , 2011, Journal of Intelligent Information Systems.

[257]  Ebad Banissi,et al.  Using font attributes in knowledge maps and information retrieval , 2014 .

[258]  Pat Hanrahan,et al.  Arc Length-Based Aspect Ratio Selection , 2011, IEEE Transactions on Visualization and Computer Graphics.

[259]  S. Johansson,et al.  Interactive Dimensionality Reduction Through User-defined Combinations of Quality Metrics , 2009, IEEE Transactions on Visualization and Computer Graphics.

[260]  Leland Wilkinson,et al.  The Grammar of Graphics (Statistics and Computing) , 2005 .

[261]  Marcus A. Magnor,et al.  Automated Analytical Methods to Support Visual Exploration of High-Dimensional Data , 2011, IEEE Transactions on Visualization and Computer Graphics.

[262]  Weidong Huang,et al.  Measuring Effectiveness of Graph Visualizations: A Cognitive Load Perspective , 2009, Inf. Vis..

[263]  Michael H. F. Wilkinson,et al.  Finding and visualizing relevant subspaces for clustering high-dimensional astronomical data using connected morphological operators , 2010, 2010 IEEE Symposium on Visual Analytics Science and Technology.

[264]  Xin Tong,et al.  TextFlow: Towards Better Understanding of Evolving Topics in Text , 2011, IEEE Transactions on Visualization and Computer Graphics.

[265]  Michael Burch,et al.  Evaluation of Traditional, Orthogonal, and Radial Tree Diagrams by an Eye Tracking Study , 2011, IEEE Transactions on Visualization and Computer Graphics.

[266]  Furu Wei,et al.  Context preserving dynamic word cloud visualization , 2010, 2010 IEEE Pacific Visualization Symposium (PacificVis).

[267]  Ulrik Brandes Force-Directed Graph Drawing , 2016, Encyclopedia of Algorithms.

[268]  Hanspeter Pfister,et al.  HiPiler: Visual Exploration of Large Genome Interaction Matrices with Interactive Small Multiples , 2017, bioRxiv.

[269]  Weiwei Cui,et al.  How Hierarchical Topics Evolve in Large Text Corpora , 2014, IEEE Transactions on Visualization and Computer Graphics.

[270]  Jürgen Döllner,et al.  Visualization of varying hierarchies by stable layout of voronoi treemaps , 2014, 2014 International Conference on Information Visualization Theory and Applications (IVAPP).

[271]  Donald W. Bouldin,et al.  A Cluster Separation Measure , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[272]  Alan M. MacEachren,et al.  How Maps Work - Representation, Visualization, and Design , 1995 .

[273]  Tamara Munzner,et al.  Visualization Analysis and Design , 2014, A.K. Peters visualization series.

[274]  Fabian Beck,et al.  Visualizing Group Structures in Graphs: A Survey , 2017, Comput. Graph. Forum.

[275]  Jessika Weiss,et al.  Vision Science Photons To Phenomenology , 2016 .

[276]  Niklas Elmqvist,et al.  Evaluating Visual Representations for Topic Understanding and Their Effects on Manually Generated Topic Labels , 2017, TACL.

[277]  Geoffrey E. Hinton,et al.  Dynamic Routing Between Capsules , 2017, NIPS.

[278]  Dirk J. Lehmann,et al.  Visualnostics: Visual Guidance Pictograms for Analyzing Projections of High‐dimensional Data , 2015, Comput. Graph. Forum.

[279]  Jürgen Döllner,et al.  Improving Layout Quality by Mixing Treemap-Layouts Based on Data-Change Characteristics , 2017, VMV.

[280]  Han-Wei Shen,et al.  Visualizing Changes of Hierarchical Data using Treemaps , 2007, IEEE Transactions on Visualization and Computer Graphics.

[281]  Daniel A. Keim,et al.  Visual Analytics of Movement , 2013, Springer Berlin Heidelberg.

[282]  Benoît Otjacques,et al.  Weighted maps: treemap visualization of geolocated quantitative data , 2015, Electronic Imaging.

[283]  Giuseppe Santucci,et al.  See What You Know: Analyzing Data Distribution to Improve Density Map Visualization , 2007, EuroVis.

[284]  Chris North,et al.  Toward measuring visualization insight , 2006, IEEE Computer Graphics and Applications.

[285]  Bernd Hamann,et al.  Progressive parallel coordinates , 2012, 2012 IEEE Pacific Visualization Symposium.

[286]  Woodie Flowers,et al.  Information content measures of visual displays , 2000, IEEE Symposium on Information Visualization 2000. INFOVIS 2000. Proceedings.