How to Assess Visual Communication of Uncertainty? A Systematic Review of Geospatial Uncertainty Visualisation User Studies

Abstract For decades, uncertainty visualisation has attracted attention in disciplines such as cartography and geographic visualisation, scientific visualisation and information visualisation. Most of this research deals with the development of new approaches to depict uncertainty visually; only a small part is concerned with empirical evaluation of such techniques. This systematic review aims to summarize past user studies and describe their characteristics and findings, focusing on the field of geographic visualisation and cartography and thus on displays containing geospatial uncertainty. From a discussion of the main findings, we derive lessons learned and recommendations for future evaluation in the field of uncertainty visualisation. We highlight the importance of user tasks for successful solutions and recommend moving towards task-centered typologies to support systematic evaluation in the field of uncertainty visualisation.

[1]  Alan M. MacEachren,et al.  Visualizing Georeferenced Data: Representing Reliability of Health Statistics , 1998 .

[2]  George L. Benwell,et al.  Assessing different approaches to visualise spatial and attribute uncertainty in socioeconomic data using the hexagonal or rhombus (HoR) trustree , 2007, Comput. Environ. Urban Syst..

[3]  Guillaume Caumon,et al.  Adjacent versus coincident representations of geospatial uncertainty: Which promote better decisions? , 2011, Comput. Geosci..

[4]  T. Hengl,et al.  Visualisation of uncertainty using the HSI colour model : computations with colours , 2003 .

[5]  George L. Benwell,et al.  Exploring Tessellation Metaphors in the Display of Geographical Uncertainty , 2008 .

[6]  Antoni Moore,et al.  The Visualisation of Uncertainty in Spatially-Referenced Attribute Data using TRUSTworthy Data Structures , 2003 .

[7]  P. Fisher.  Visualizing Uncertainty in Soil Maps by Animation , 1993 .

[8]  Limor Nadav-Greenberg,et al.  The Effect of Uncertainty Visualizations on Decision Making in Weather Forecasting , 2008 .

[9]  Tobias Isenberg,et al.  Evaluating Sketchiness as a Visual Variable for the Depiction of Qualitative Uncertainty , 2012, IEEE Transactions on Visualization and Computer Graphics.

[10]  Alex T. Pang,et al.  Glyphs for Visualizing Uncertainty in Vector Fields , 1996, IEEE Trans. Vis. Comput. Graph..

[11]  M. Sheelagh T. Carpendale,et al.  Theoretical analysis of uncertainty visualizations , 2006, Electronic Imaging.

[12]  Timothy S. Newman,et al.  On visualizing uncertainty in volumetric data: techniques and their evaluation , 2004, J. Vis. Lang. Comput..

[13]  Maria Riveiro.  Cognitive evaluation of uncertainty visualization methods for decision making , 2007, APGV.

[14]  Ari Kolbeinsson,et al.  Visualising uncertainty in aircraft cockpits : Is icon degradation an appropriate visualisation form , 2013 .

[15]  Lucy Bastin,et al.  Evaluating the Perception of Uncertainty in Alternative Visualization Strategies , 2000, Cartogr. Int. J. Geogr. Inf. Geovisualization.

[16]  J. Llinas,et al.  Human Performance and Data Fusion Based Decision Aids , 1999 .

[17]  Ken Brodlie,et al.  A Review of Uncertainty in Data Visualization , 2012, Expanding the Frontiers of Visual Analytics and Visualization.

[18]  Adrienne Grêt-Regamey,et al.  Visualization of uncertainty in natural hazards assessments using an interactive cartographic information system , 2011 .

[19]  Keith C. Clarke,et al.  Testing Popular Visualization Techniques for Representing Model Uncertainty , 2003 .

[20]  Christoph Kinkeldey,et al.  Beyond the Surface: Current Issues and Future Directions in Uncertainty Visualization Research , 2013 .

[21]  Beverley J. Evans,et al.  Dynamic display of spatial data-reliability: does it benefit the map user? , 1997 .

[22]  Richard T. Stone,et al.  Visual Representations of Meta-Information , 2009 .

[23]  Robert J. Moorhead,et al.  A User Study to Compare Four Uncertainty Visualization Methods for 1D and 2D Datasets , 2009, IEEE Transactions on Visualization and Computer Graphics.

[24]  Penny Rheingans,et al.  Point-based probabilistic surfaces to show surface uncertainty , 2004, IEEE Transactions on Visualization and Computer Graphics.

[25]  Mark Gahegan,et al.  Visualizing Geospatial Information Uncertainty: What We Know and What We Need to Know , 2005 .

[26]  Penny Rheingans,et al.  Procedural annotation of uncertain information , 2000, Proceedings Visualization 2000. VIS 2000 (Cat. No.00CH37145).

[27]  Alan M. MacEachren,et al.  VISUALIZING UNCERTAIN INFORMATION , 1992 .

[28]  Daniel C. Cliburn,et al.  Evaluating the Usability of a Tool for Visualizing the Uncertainty of the Future Global Water Balance , 2003 .

[29]  Andrew Mercer,et al.  Noodles: A Tool for Visualization of Numerical Weather Model Ensemble Uncertainty , 2010, IEEE Transactions on Visualization and Computer Graphics.

[30]  Alexander Klippel,et al.  Evaluation of noise annotation lines: using noise to represent thematic uncertainty in maps , 2014 .

[31]  C. A. Blok,et al.  Animated representation of uncertainty and fuzziness in Dutch spatial planning maps , 2008 .

[32]  Robert M. Edsall,et al.  The Influence of Uncertainty Visualization on Decision Making: An Empirical Evaluation , 2006 .

[33]  Elisabeth S. Nelson,et al.  Visualizing Data Certainty: A Case Study Using Graduated Circle Maps , 2001 .

[34]  Michael F. Goodchild,et al.  Visualizing spatial data uncertainty using animation , 1997 .

[35]  D. Retchless.  Mapping Climate Change Uncertainty: Effects on Risk Perceptions and Decision Making , 2012 .

[36]  Nahum D. Gershon.  Visualization of an Imperfect World , 1998, IEEE Computer Graphics and Applications.

[37]  Alex T. Pang,et al.  Approaches to uncertainty visualization , 1996, The Visual Computer.

[38]  Igor Drecki,et al.  Visualisation of Uncertainty in Geographical Data , 2002 .

[39]  Mark Gahegan,et al.  A typology for visualizing uncertainty , 2005, IS&T/SPIE Electronic Imaging.

[40]  K. Alberti,et al.  Web-based visualization of uncertain spatio-temporal data , 2014 .

[41]  B. Buttenfield,et al.  Guidelines for the Display of Attribute Certainty , 2000 .

[42]  Robert S. Laramee,et al.  Uncertainty Visualization Methods in Isosurface Volume Rendering , 2002 .

[43]  Mathew A. Dooley,et al.  Visualizing Method-Produced Uncertainty in Isometric Mapping , 2007 .

[44]  J. Flannery.  THE RELATIVE EFFECTIVENESS OF SOME COMMON GRADUATED POINT SYMBOLS IN THE PRESENTATION OF QUANTITATIVE DATA , 1971 .

[45]  Ken Brodlie,et al.  Contouring with Uncertainty , 2008, TPCG.

[46]  J. D. Bulens,et al.  Dealing with the Uncertainty of Having Incomplete Sources of Geo-Information in Spatial Planning , 2013 .

[47]  Mark Gahegan,et al.  Visual Semiotics & Uncertainty Visualization: An Empirical Study , 2012, IEEE Transactions on Visualization and Computer Graphics.

[48]  Ann M. Bisantz,et al.  Utilizing Graphical Formats to Convey Uncertainty in a Decision Making Task , 2000 .

[49]  Angela Schwering,et al.  Usability of Spatio-Temporal Uncertainty Visualisation Methods , 2012, AGILE Conf..

[50]  Maria Riveiro,et al.  Evaluation of uncertainty visualization techniques for information fusion , 2007, 2007 10th International Conference on Information Fusion.

[51]  D. Peuquet.  It's About Time: A Conceptual Framework for the Representation of Temporal Dynamics in Geographic Information Systems , 1994 .

[52]  David Howard,et al.  Interface Design for Geographic Visualization: Tools for Representing Reliability , 1996 .

[53]  Petr Kubícek,et al.  Thematic uncertainty visualization usability – comparison of basic methods , 2011, Ann. GIS.

[54]  Monica M. C. Schraefel,et al.  Trust me, i'm partially right: incremental visualization lets analysts explore large datasets faster , 2012, CHI.

[55]  Herbert A. Simon,et al.  Why a Diagram is (Sometimes) Worth Ten Thousand Words , 1987, Cogn. Sci..