Towards a visual guide for communicating uncertainty in Visual Analytics

Abstract This article presents a first step towards the definition of a visual guide for communicating uncertainty which is to fit into existing visualisation frameworks and toolkits. The first entry in our guide is made by a set of visual variables appropriate for representing areal uncertainty in algorithm mechanics. Such visualisations show users how data points are distributed in the classification space and allow them to understand the “goodness-of-fit” of their data to the algorithm. This is important for Visual Analytics applications, which combine Information Visualisation with information mining techniques in an interactive decision-making process. Model uncertainties stemming from widely spread data points need to be visualised so that the user can make adjustments and improve the analysis. To capitalise on established knowledge and meaning, we explore whether popular visual variables for representing areal uncertainty in the domain of geospatial visualisation may also be effective for representing uncertainty in the visualisation of the mechanics of K-means clustering and Linear Regression algorithms, as both use a spatial distribution of data points. In a study with 500 participants we find that overall the visual means opacity performs best, followed by texture, but that grid and blur may be unsuitable for quantifying uncertainty. The performance of contour lines appears to depend on the algorithm visualisation. Using this study, we extend the validity of a set of domain-specific findings from geospatial visualisation to the visualisation of algorithm mechanics and use these to form the first building blocks of a cross-disciplinary visual guide for representing uncertainty, laying promising foundations for future work.

[1]  P. Fisher Visualizing Uncertainty in Soil Maps by Animation , 1993 .

[2]  Iván Martínez-Ortiz,et al.  A visual language for the creation of narrative educational games , 2011, J. Vis. Lang. Comput..

[3]  Christopher Andrews,et al.  The human is the loop: new directions for visual analytics , 2014, Journal of Intelligent Information Systems.

[4]  Eliseo Clementini,et al.  A model for uncertain lines , 2005, J. Vis. Lang. Comput..

[5]  U. Ziegler,et al.  The flowchart interpreter for introductory programming courses , 1998, FIE '98. 28th Annual Frontiers in Education Conference. Moving from 'Teacher-Centered' to 'Learner-Centered' Education. Conference Proceedings (Cat. No.98CH36214).

[6]  Lodewijk Bergmans,et al.  Vibes: A visual language for specifying behavioral requirements of algorithms , 2013, J. Vis. Lang. Comput..

[7]  Lucy Bastin,et al.  Visualizing uncertainty in multi-spectral remotely sensed imagery , 2002 .

[8]  Paul Rosen,et al.  From Quantification to Visualization: A Taxonomy of Uncertainty Visualization Approaches , 2011, WoCoUQ.

[9]  Alan F. Blackwell,et al.  Pictorial Representation and Metaphor in Visual Language Design , 2001, J. Vis. Lang. Comput..

[10]  Kang Zhang,et al.  Enabling decision trend analysis with interactive scatter plot matrices visualization , 2016, J. Vis. Lang. Comput..

[11]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[12]  Rashmi R. Sinha,et al.  The role of transparency in recommender systems , 2002, CHI Extended Abstracts.

[13]  Tobias Isenberg,et al.  Evaluating Sketchiness as a Visual Variable for the Depiction of Qualitative Uncertainty , 2012, IEEE Transactions on Visualization and Computer Graphics.

[14]  David J Spiegelhalter,et al.  Funnel plots for comparing institutional performance , 2005, Statistics in medicine.

[15]  Daniel A. Keim,et al.  Mastering the Information Age - Solving Problems with Visual Analytics , 2010 .

[16]  Ola Ahlqvist,et al.  Representing and negotiating uncertain geospatial concepts - Where are the exurban areas? , 2009, Comput. Environ. Urban Syst..

[17]  Timothy S. Newman,et al.  On visualizing uncertainty in volumetric data: techniques and their evaluation , 2004, J. Vis. Lang. Comput..

[18]  David S. Ebert,et al.  FinVis: Applied visual analytics for personal financial planning , 2009, 2009 IEEE Symposium on Visual Analytics Science and Technology.

[19]  Alan M. MacEachren,et al.  VISUALIZING UNCERTAIN INFORMATION , 1992 .

[20]  Arko Lucieer,et al.  Interactive and visual fuzzy classification of remotely sensed imagery for exploration of uncertainty , 2004, Int. J. Geogr. Inf. Sci..

[21]  Robert J. Moorhead,et al.  A User Study to Compare Four Uncertainty Visualization Methods for 1D and 2D Datasets , 2009, IEEE Transactions on Visualization and Computer Graphics.

[22]  Barbara P. Buttenfield,et al.  Mapping Ecological Uncertainty , 2001 .

[23]  Baltasar Fernández-Manjón,et al.  A narrative metaphor to facilitate educational game authoring , 2012, Comput. Educ..

[24]  Aniket Kittur,et al.  Crowdsourcing user studies with Mechanical Turk , 2008, CHI.

[25]  Trevor Paterson,et al.  Visual cleaning of genotype data , 2013, 2013 IEEE Symposium on Biological Data Visualization (BioVis).

[26]  Jürgen Bernard,et al.  Visual-Interactive Preprocessing of Time Series Data , 2012, SIGRAD.

[27]  Daniel Perry,et al.  VizDeck: a card game metaphor for fast visual data exploration , 2012, CHI EA '12.

[28]  Augusto Celentano,et al.  From real to metaphoric maps: Cartography as a visual language for organizing and sharing knowledge , 2012, J. Vis. Lang. Comput..

[29]  George Siemens,et al.  Penetrating the fog: analytics in learning and education , 2014 .

[30]  Alex Pang,et al.  Visualizing Uncertainty in Geo-spatial Data , 2001 .

[31]  T. Hengl,et al.  Visualisation of uncertainty using the HSI colour model : computations with colours , 2003 .

[32]  Martyn Jessop Digital visualization as a scholarly activity , 2008, Lit. Linguistic Comput..

[33]  Ioan Alfred Letia,et al.  Model checking as support for inspecting compliance to rules in flexible processes , 2015, J. Vis. Lang. Comput..

[34]  Jason Dykes,et al.  Exploring Uncertainty in Geodemographics with Interactive Graphics , 2011, IEEE Transactions on Visualization and Computer Graphics.

[35]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[36]  Dominique Brodbeck,et al.  Research directions in data wrangling: Visualizations and transformations for usable and credible data , 2011, Inf. Vis..

[37]  Mark Gahegan,et al.  Visualizing Geospatial Information Uncertainty: What We Know and What We Need to Know , 2005 .

[38]  Siti Salwah Salim,et al.  A systematic review of shared visualisation to achieve common ground , 2015, J. Vis. Lang. Comput..

[39]  Anastasia Bezerianos,et al.  Exploration Views: Understanding Dashboard Creation and Customization for Visualization Novices , 2011, INTERACT.

[40]  Yu Liu,et al.  Probability issues in locality descriptions based on Voronoi neighbor relationship , 2012, J. Vis. Lang. Comput..

[41]  Daniel A. Keim,et al.  The Role of Uncertainty, Awareness, and Trust in Visual Analytics , 2016, IEEE Transactions on Visualization and Computer Graphics.

[42]  Peter A. Burrough,et al.  Natural Objects with Indeterminate Boundaries , 2020 .

[43]  Hubert Cuyckens,et al.  The Oxford handbook of cognitive linguistics , 2010 .

[44]  David Howard,et al.  Interface Design for Geographic Visualization: Tools for Representing Reliability , 1996 .

[45]  Slava Kisilevich,et al.  A conceptual framework and taxonomy of techniques for analyzing movement , 2011, J. Vis. Lang. Comput..

[46]  Jing Wang,et al.  A Framework for Quality Assurance in Crowdsourcing , 2013 .

[47]  Daniel C. Cliburn,et al.  Evaluating the Usability of a Tool for Visualizing the Uncertainty of the Future Global Water Balance , 2003 .

[48]  Alexander Klippel,et al.  Evaluation of noise annotation lines: using noise to represent thematic uncertainty in maps , 2014 .

[49]  Kanit Wongsuphasawat,et al.  Voyager: Exploratory Analysis via Faceted Browsing of Visualization Recommendations , 2016, IEEE Transactions on Visualization and Computer Graphics.

[50]  Moslem Yousefi,et al.  Flowchart-based Bayesian Intelligent Tutoring System for computer programming , 2015, 2015 International Conference on Smart Sensors and Application (ICSSA).

[51]  Bernd Meyer,et al.  Visual Language Theory , 2012, Springer New York.

[52]  Brian D. Fisher,et al.  Visual analytic roadblocks for novice investigators , 2011, 2011 IEEE Conference on Visual Analytics Science and Technology (VAST).

[53]  Jeffrey Heer,et al.  Profiler: integrated statistical analysis and visualization for data quality assessment , 2012, AVI.

[54]  Alex T. Pang,et al.  Visualizing scalar volumetric data with uncertainty , 2002, Comput. Graph..

[55]  Giuseppe Santucci,et al.  Improving visual analytics environments through a methodological framework for automatic clutter reduction , 2011, J. Vis. Lang. Comput..

[56]  Lisa Gralewski,et al.  Theory and Practice of Computer Graphics , 2004 .

[57]  M. Sheelagh T. Carpendale,et al.  Personal Visualization and Personal Visual Analytics , 2015, IEEE Transactions on Visualization and Computer Graphics.

[58]  Erik Duval,et al.  Studying the history of philosophical ideas: supporting research discovery, navigation, and awareness , 2014, i-KNOW '14.

[59]  Bongshin Lee,et al.  Revealing Uncertainty for Information Visualization , 2010, Inf. Vis..

[60]  M. Leitner CARTOGRAPHIC GUIDELINES ON THE VISUALIZATION OF ATTRIBUTE ACCURACY , 2008 .

[61]  Tim Dwyer,et al.  Untangling Euler Diagrams , 2010, IEEE Transactions on Visualization and Computer Graphics.

[62]  James J. Thomas,et al.  Challenges for Visual Analytics , 2009, Inf. Vis..

[63]  Kristin A. Cook,et al.  Illuminating the Path: The Research and Development Agenda for Visual Analytics , 2005 .

[64]  Christine L. Borgman,et al.  The Digital Future is Now: A Call to Action for the Humanities , 2009, Digit. Humanit. Q..

[65]  Alan F. Blackwell,et al.  The reification of metaphor as a design tool , 2006, TCHI.

[66]  Uniuersita di L'Aquila An Algebraic Model for Spatial Objects with Indeterminate Boundaries , 2012 .

[67]  W. Mackaness,et al.  Visual access to data quality in geographic information systems , 1993 .

[68]  Elisabeth S. Nelson,et al.  Visualizing Data Certainty: A Case Study Using Graduated Circle Maps , 2001 .

[69]  Fabien Duchateau,et al.  Uncertainty visualization of multi-providers cartographic integration , 2014, J. Vis. Lang. Comput..

[70]  Zhen Sun,et al.  A Small Target Detection Method based on Human Visual System and Confidence Measurement , 2016, J. Inf. Hiding Multim. Signal Process..

[71]  Jacob L. Cybulski,et al.  Metaphors in Interactive Visual Analytics , 2014, VINCI '14.

[72]  Steven P. French,et al.  Risk assessment, modeling and decision support : strategic directions , 2007 .

[73]  Alan M. MacEachren,et al.  How to Assess Visual Communication of Uncertainty? A Systematic Review of Geospatial Uncertainty Visualisation User Studies , 2014 .

[74]  Lei Ren,et al.  DaisyViz: A model-based user interface toolkit for interactive information visualization systems , 2010, J. Vis. Lang. Comput..

[75]  Jarke J. van Wijk,et al.  Challenges for Visual Analytics , 2017, VISIGRAPP.

[76]  Ken Brodlie,et al.  Uncertain Flow Visualization using LIC , 2009, TPCG.

[77]  Martin Erwig,et al.  A visual language for explaining probabilistic reasoning , 2013, J. Vis. Lang. Comput..

[78]  John T. Stasko,et al.  Toward a Deeper Understanding of the Role of Interaction in Information Visualization , 2007, IEEE Transactions on Visualization and Computer Graphics.

[79]  Philippe Castagliola,et al.  A Comparison of the Readability of Graphs Using Node-Link and Matrix-Based Representations , 2004, IEEE Symposium on Information Visualization.

[80]  Kang Zhang Using visual languages in management , 2012, J. Vis. Lang. Comput..

[81]  Shwetak N. Patel,et al.  How Good is 85%?: A Survey Tool to Connect Classifier Evaluation to Acceptability of Accuracy , 2015, CHI.

[82]  Grayna lusarczyk,et al.  Visual language and graph-based structures in conceptual design , 2012 .

[83]  K. Andolsek,et al.  Risk assessment , 2003, Nature.

[84]  Ken Brodlie,et al.  A Review of Uncertainty in Data Visualization , 2012, Expanding the Frontiers of Visual Analytics and Visualization.

[85]  Dieter W. Fellner,et al.  Trajectory-based visual analysis of large financial time series data , 2007, SKDD.

[86]  Daniela Fogli,et al.  Visual Interactive Systems for End-User Development: A Model-Based Design Methodology , 2007, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[87]  Kelly Rose,et al.  Variable Grid Method: An Intuitive Approach for Simultaneously Quantifying and Visualizing Spatial Data and Uncertainty , 2015, Trans. GIS.

[88]  Mark Gahegan,et al.  Visual Semiotics & Uncertainty Visualization: An Empirical Study , 2012, IEEE Transactions on Visualization and Computer Graphics.

[89]  Daniel A. Keim,et al.  Spatiotemporal Analysis of Sensor Logs using Growth Ring Maps , 2009, IEEE Transactions on Visualization and Computer Graphics.

[90]  Nahum D. Gershon,et al.  Visualization of fuzzy data using generalized animation , 1992, Proceedings Visualization '92.

[91]  Penny Rheingans,et al.  Procedural annotation of uncertain information , 2000, Proceedings Visualization 2000. VIS 2000 (Cat. No.00CH37145).

[92]  Andrew Mercer,et al.  Noodles: A Tool for Visualization of Numerical Weather Model Ensemble Uncertainty , 2010, IEEE Transactions on Visualization and Computer Graphics.

[93]  Angela Schwering,et al.  Usability of Spatio-Temporal Uncertainty Visualisation Methods , 2012, AGILE Conf..

[94]  Steven Hansen,et al.  Designing Educationally Effective Algorithm Visualizations , 2002, J. Vis. Lang. Comput..

[95]  Daniel A. Keim,et al.  Visual Analytics: Scope and Challenges , 2008, Visual Data Mining.

[96]  Pak Chung Wong,et al.  Expanding the Frontiers of Visual Analytics and Visualization , 2012, Springer London.

[97]  C. Peter Keller,et al.  Modelling and visualizing multiple spatial uncertainties , 1997 .

[98]  Daniel C. Cliburn,et al.  Design and evaluation of a decision support system in a water balance application , 2002, Comput. Graph..