Public Understanding of Visual Representations of Uncertainty in Temperature Forecasts

Multiday weather forecasts often include graphical representations of uncertainty. However, visual representations of probabilistic events are often misinterpreted by the general public. Although various uncertainty visualizations are now in use, the parameters that determine their successful deployment are still unknown. At the same time, a correct understanding of possible weather forecast outcomes will enable the public to make better decisions and will increase their trust in these predictions. We investigated the effects of the visual form and width of temperature forecast visualizations with uncertainty on estimates of the probability that the temperature could exceed a given value. The results suggest that people apply an internal model of the uncertainty distribution that closely resembles a normal distribution, which confirms previous findings. Also, the visualization type appears to affect the applied internal model, in particular the probability estimates of values outside the depicted uncertainty range. Furthermore, we find that perceived uncertainty does not necessarily map linearly to visual features, as identical relative positions to the range are being judged differently depending on the width of the uncertainty range. Finally, the internal model of the uncertainty distribution is related to participants’ numeracy. We include some implications for makers or designers of uncertainty visualizations.

[1]  S. Joslyn,et al.  The Advantages of Predictive Interval Forecasts for Non‐Expert Users and the Impact of Visualizations , 2013 .

[2]  Sam Huf,et al.  Visualizing Uncertainty to Improve Operators' Spatial Proximity Judgments in Uncertain Surroundings , 2006 .

[3]  Robert J. Moorhead,et al.  A User Study to Compare Four Uncertainty Visualization Methods for 1D and 2D Datasets , 2009, IEEE Transactions on Visualization and Computer Graphics.

[4]  Rebecca E. Morss,et al.  Examining the use of weather forecasts in decision scenarios: results from a US survey with implications for uncertainty communication , 2010 .

[5]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[6]  Limor Nadav-Greenberg,et al.  Reducing probabilistic weather forecasts to the worst-case scenario: anchoring effects. , 2011, Journal of experimental psychology. Applied.

[7]  Marcel Fratzscher,et al.  Low probability, high impact: Policy making and extreme events , 2008 .

[8]  George C. Craig Probabilistic Weather Forecasting , 2012 .

[9]  ZOE HILDON,et al.  Impact of format and content of visual display of data on comprehension, choice and preference: a systematic review. , 2012, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[10]  Alex T. Pang,et al.  Approaches to uncertainty visualization , 1996, The Visual Computer.

[11]  Pricilla Marimo,et al.  Communication of uncertainty in weather forecasts , 2012 .

[12]  Geoff Cumming,et al.  Inference by Eye: Pictures of Confidence Intervals and Thinking About Levels of Confidence , 2007 .

[13]  Gary E. Bolton,et al.  A Laboratory Study of the Benefits of Including Uncertainty Information in Weather Forecasts , 2006 .

[14]  Anthony Leiserowitz,et al.  Misinterpretations of the “Cone of Uncertainty” in Florida during the 2004 Hurricane Season , 2007 .

[15]  A. H. Murphy,et al.  Misinterpretations of precipitation probability forecasts , 1980 .

[16]  M. Granger Morgan,et al.  Graphical Communication of Uncertain Quantities to Nontechnical People , 1987 .

[17]  Marie Juanchich,et al.  Improbable outcomes: Infrequent or extraordinary? , 2013, Cognition.

[18]  P. Ubel,et al.  Measuring Numeracy without a Math Test: Development of the Subjective Numeracy Scale , 2007, Medical decision making : an international journal of the Society for Medical Decision Making.

[19]  Beverley J. Evans,et al.  Dynamic display of spatial data-reliability: does it benefit the map user? , 1997 .

[20]  Richard T. Stone,et al.  Visual Representations of Meta-Information , 2009 .

[21]  Karl Halvor Teigen,et al.  Can > Will: Predictions of What Can Happen are Extreme, but Believed to be Probable , 2013 .

[22]  Susan L Joslyn,et al.  Uncertainty forecasts improve weather-related decisions and attenuate the effects of forecast error. , 2012, Journal of experimental psychology. Applied.

[23]  Lieven Verschaffel,et al.  The Heuristic Interpretation of Box Plots. , 2013 .

[24]  Gerd Gigerenzer,et al.  “A 30% Chance of Rain Tomorrow”: How Does the Public Understand Probabilistic Weather Forecasts? , 2005, Risk analysis : an official publication of the Society for Risk Analysis.

[25]  Elisabeth Stephens,et al.  Communicating probabilistic information from climate model ensembles—lessons from numerical weather prediction , 2012 .

[26]  Marie Juanchich,et al.  Verbal Probabilities: An Alternative Approach , 2014, Quarterly journal of experimental psychology.

[27]  G. Cumming,et al.  Researchers misunderstand confidence intervals and standard error bars. , 2005, Psychological methods.

[28]  Anthony D. Andre,et al.  Displaying Uncertainty in Advanced Navigation Systems , 1998 .

[29]  Penny Rheingans,et al.  Procedural annotation of uncertain information , 2000, Proceedings Visualization 2000. VIS 2000 (Cat. No.00CH37145).

[30]  Andrew Mercer,et al.  Noodles: A Tool for Visualization of Numerical Weather Model Ensemble Uncertainty , 2010, IEEE Transactions on Visualization and Computer Graphics.

[31]  M. Sheelagh T. Carpendale,et al.  Visualization of Uncertainty and Reasoning , 2007, Smart Graphics.

[32]  S. Joslyn,et al.  Decisions With Uncertainty: The Glass Half Full , 2013 .

[33]  M. Galesic,et al.  Who profits from visual aids: overcoming challenges in people's understanding of risks [corrected]. , 2010, Social science & medicine.

[34]  Marie Juanchich,et al.  Top scores are possible, bottom scores are certain (and middle scores are not worth mentioning): A pragmatic view of verbal probabilities , 2013, Judgment and Decision Making.

[35]  Rebecca E. Morss,et al.  How forecasts expressing uncertainty are perceived by UK students , 2013, Weather.

[36]  Hans-Christian Hege,et al.  Positional Uncertainty of Isocontours: Condition Analysis and Probabilistic Measures , 2011, IEEE Transactions on Visualization and Computer Graphics.

[37]  Susan Joslyn,et al.  Visualizing temperature forecast uncertainty for a non-expert web audience , 2009 .

[38]  Paul Rosen,et al.  From Quantification to Visualization: A Taxonomy of Uncertainty Visualization Approaches , 2011, WoCoUQ.

[39]  Alex Kirlik Lessons Learned from the Design of the Decision Support System Used in the Hurricane Katrina Evacuation Decision , 2007 .

[40]  H. Griethe Visualizing Uncertainty for Improved Decision Making , 2005 .

[41]  B. Newell Judgment Under Uncertainty , 2013 .

[42]  Limor Nadav-Greenberg,et al.  The Effect of Uncertainty Visualizations on Decision Making in Weather Forecasting , 2008 .

[43]  Michèle M. M. Mazzocco,et al.  Inferring uncertainty from interval estimates: Effects of alpha level and numeracy , 2013, Judgment and Decision Making.

[44]  Alexander Toet,et al.  The perception of visual uncertainty representation by non-experts , 2013 .

[45]  Limor Nadav-Greenberg,et al.  Uncertainty Forecasts Improve Decision Making Among Nonexperts , 2009 .

[46]  C. Leith Theoretical Skill of Monte Carlo Forecasts , 1974 .

[47]  Nahum D. Gershon Visualization of an Imperfect World , 1998, IEEE Computer Graphics and Applications.

[48]  Maria Riveiro,et al.  Evaluation of uncertainty visualization techniques for information fusion , 2007, 2007 10th International Conference on Information Fusion.

[49]  Michael F. Goodchild,et al.  Visualizing spatial data uncertainty using animation , 1997 .

[50]  Jutta Thielen,et al.  Ensemble predictions and perceptions of risk, uncertainty, and error in flood forecasting , 2007 .

[51]  Susan Joslyn,et al.  Communicating forecast uncertainty: public perception of weather forecast uncertainty , 2010 .

[52]  Adrian E. Raftery,et al.  Weather Forecasting with Ensemble Methods , 2005, Science.

[53]  Mark Gahegan,et al.  Visualizing Geospatial Information Uncertainty: What We Know and What We Need to Know , 2005 .

[54]  Alexander Toet,et al.  The Perception of Visual UncertaintyRepresentation by Non-Experts , 2014, IEEE Transactions on Visualization and Computer Graphics.

[55]  Ross T. Whitaker,et al.  Contour Boxplots: A Method for Characterizing Uncertainty in Feature Sets from Simulation Ensembles , 2013, IEEE Transactions on Visualization and Computer Graphics.

[56]  Ulrich Hoffrage,et al.  Visual representation of statistical information improves diagnostic inferences in doctors and their patients. , 2013, Social science & medicine.

[57]  F. Pappenberger,et al.  The European Flood Alert System and the communication, perception, and use of ensemble predictions for operational flood risk management , 2013 .

[58]  Mark S. Roulston,et al.  A laboratory‐based study of understanding of uncertainty in 5‐day site‐specific temperature forecasts , 2009 .

[59]  Tobias Isenberg,et al.  Evaluating Sketchiness as a Visual Variable for the Depiction of Qualitative Uncertainty , 2012, IEEE Transactions on Visualization and Computer Graphics.

[60]  Florian Pappenberger,et al.  Do probabilistic forecasts lead to better decisions , 2012 .

[61]  Rebecca E. Morss,et al.  Communicating Uncertainty in Weather Forecasts: A Survey of the U.S. Public , 2008 .

[62]  Susan Joslyn,et al.  The Benefits and Challenges of Predictive Interval Forecasts and Verification Graphics for End Users , 2013 .

[63]  Ann M. Bisantz,et al.  Utilizing Graphical Formats to Convey Uncertainty in a Decision Making Task , 2000 .

[64]  Ecmwf Newsletter,et al.  EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS , 2004 .

[65]  Ramón de Elía,et al.  Diversity in Interpretations of Probability: Implications for Weather Forecasting , 2005 .

[66]  Mike Pearson,et al.  Visualizing Uncertainty About the Future , 2022 .