Examining the use of weather forecasts in decision scenarios: results from a US survey with implications for uncertainty communication

The hydrometeorological community has limited understanding of how people interpret forecast information and use it in decision making, hampering effective forecast communication. This article addresses these issues in the context of weather prediction, focusing especially on forecast uncertainty. It does so using empirical data from decision scenario questions asked in a nationwide US survey. Respondents were asked their probabilistic threshold for taking action to protect against potential rain or frost. They were then asked to make yes/no protective decisions in a potential reservoir flooding or fruit frost scenario given different forecasts. The results indicate that people have different probabilistic thresholds for taking protective action and that context and presentation influence forecast use. The results also suggest that many people infer uncertainty into deterministic forecasts, and that many respondents were able to interpret probabilistic forecasts of the type presented well enough to use them in the decision questions. Further, the analysis suggests that most respondents did not make decisions according to the simplest form of the cost-loss decision model. The analysis also examines relationships between respondents' information use and other aspects of their perceptions and interpretations of forecast uncertainty, including their interpretations of probability of precipitation. The findings add to fundamental knowledge about people's interpretations and use of weather forecasts, especially forecasts that explicitly convey uncertainty, and provide a starting point for future related work using survey and experimental approaches. Copyright © 2010 Royal Meteorological Society

[1]  John Handmer,et al.  Communicating uncertainty via probabilities: The case of weather forecasts , 2007 .

[2]  B. Rimer,et al.  General Performance on a Numeracy Scale among Highly Educated Samples , 2001, Medical decision making : an international journal of the Society for Medical Decision Making.

[3]  U. Fischbacher,et al.  A Nation-Wide Laboratory: Examining Trust and Trustworthiness by Integrating Behavioral Experiments into Representative Survey , 2003, SSRN Electronic Journal.

[4]  John D. Hey Does Repetition Improve Consistency? , 2001 .

[5]  J. Kagel,et al.  Handbook of Experimental Economics , 1997 .

[6]  R. Hertwig,et al.  Experimental practices in economics: A methodological challenge for psychologists? , 2001, Behavioral and Brain Sciences.

[7]  A. O'connell Logistic Regression Models for Ordinal Response Variables , 2005 .

[8]  J. C. Thompson,et al.  On the Operational Deficiences in Categorical Weather Forecasts , 1952 .

[9]  Michael J. Brennan,et al.  Operational Forecaster Uncertainty Needs and Future Roles , 2008 .

[10]  Kenneth R. Mylne,et al.  Decision‐making from probability forecasts based on forecast value , 2002 .

[11]  Elke U. Weber,et al.  The Interpretation of 'Likely' Depends on the Context, But 70% is 70% - Right? The Influence of Associative Processes on Perceived Certainty , 1999 .

[12]  Mark S. Roulston,et al.  A laboratory‐based study of understanding of uncertainty in 5‐day site‐specific temperature forecasts , 2009 .

[13]  J. Krosnick Response strategies for coping with the cognitive demands of attitude measures in surveys , 1991 .

[14]  Robert T. Ryan Digital Forecasts - Communication, Public Understanding, and Decision Making , 2003 .

[15]  冨山 芳幸,et al.  「Completing the Forecast : Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts」, National Research Council著, The National Academies Press, 2006年, 112頁, 25.88ドル, ISBN-13:978-0-309-10255-1(本だな) , 2007 .

[16]  Gary A. Ballinger,et al.  Using Generalized Estimating Equations for Longitudinal Data Analysis , 2004 .

[17]  S. Presser,et al.  Questions and Answers in Attitude Surveys: Experiments on Question Form, Wording, and Context , 1996 .

[18]  A. Tversky,et al.  Prospect theory: an analysis of decision under risk — Source link , 2007 .

[19]  Schneider,et al.  All Frames Are Not Created Equal: A Typology and Critical Analysis of Framing Effects. , 1998, Organizational behavior and human decision processes.

[20]  A. Tversky,et al.  Advances in prospect theory: Cumulative representation of uncertainty , 1992 .

[21]  Limor Nadav-Greenberg,et al.  Probability of Precipitation: Assessment and Enhancement of End-User Understanding , 2009 .

[22]  Donald Waldman,et al.  Household Evacuation Decision Making and the Benefits of Improved Hurricane Forecasting: Developing a Framework for Assessment , 2009 .

[23]  D. Richardson Skill and relative economic value of the ECMWF ensemble prediction system , 2000 .

[24]  J. C. Thompson,et al.  THE ECONOMIC UTILITY OF WEATHER FORECASTS , 1955 .

[25]  L. Rips,et al.  The Psychology of Survey Response , 2000 .

[26]  Colin Camerer,et al.  The Effects of Financial Incentives in Experiments: A Review and Capital-Labor-Production Framework , 1999 .

[27]  Gerd Gigerenzer,et al.  “A 30% Chance of Rain Tomorrow”: How Does the Public Understand Probabilistic Weather Forecasts? , 2005, Risk analysis : an official publication of the Society for Risk Analysis.

[28]  Limor Nadav-Greenberg,et al.  The Effect of Uncertainty Visualizations on Decision Making in Weather Forecasting , 2008 .

[29]  D. Dillman Mail and internet surveys: The tailored design method, 2nd ed. , 2007 .

[30]  J. Neumann,et al.  Theory of games and economic behavior , 1945, 100 Years of Math Milestones.

[31]  Tim N. Palmer,et al.  The economic value of ensemble forecasts as a tool for risk assessment: From days to decades , 2002 .

[32]  Colin Camerer An experimental test of several generalized utility theories , 1989 .

[33]  Rachel Croson,et al.  The Method of Experimental Economics , 2005 .

[34]  David S. Richardson,et al.  ON THE ECONOMIC VALUE OF ENSEMBLE BASED WEATHER FORECASTS , 2001 .

[35]  Martin R. Manning,et al.  The Difficulty of Communicating Uncertainty , 2003 .

[36]  Robert Sugden,et al.  Probability and juxtaposition effects: An experimental investigation of the common ratio effect , 1989 .

[37]  Rebecca E. Morss,et al.  Communicating Uncertainty in Weather Forecasts: A Survey of the U.S. Public , 2008 .

[38]  Susan Joslyn,et al.  Probability or frequency? Expressing forecast uncertainty in public weather forecasts , 2009 .

[39]  C. K. Mertz,et al.  PSYCHOLOGICAL SCIENCE Research Article Numeracy and Decision Making , 2022 .

[40]  David V. Budescu,et al.  Improving Communication of Uncertainty in the Reports of the Intergovernmental Panel on Climate Change , 2009, Psychological science.

[41]  Pavlo R. Blavatskyy,et al.  Stochastic expected utility theory , 2005 .

[42]  A. Kühberger,et al.  The Influence of Framing on Risky Decisions: A Meta-analysis. , 1998, Organizational behavior and human decision processes.

[43]  Susan Joslyn,et al.  The Effect of Probabilistic Information on Threshold Forecasts , 2007 .

[44]  Limor Nadav-Greenberg,et al.  The effects of wording on the understanding and use of uncertainty information in a threshold forecasting decision , 2009 .

[45]  A. Tversky,et al.  Prospect Theory : An Analysis of Decision under Risk Author ( s ) : , 2007 .

[46]  Paul D. Allison,et al.  Logistic Regression Using the SAS System : Theory and Application , 1999 .

[47]  Jutta Thielen,et al.  Ensemble predictions and perceptions of risk, uncertainty, and error in flood forecasting , 2007 .

[48]  George Loewenstein,et al.  Experimental Economics From the Vantage‐point of Behavioural Economics , 1999 .

[49]  Rebecca E. Morss,et al.  Flood Risk, Uncertainty, and Scientific Information for Decision Making: Lessons from an Interdisciplinary Project , 2005 .

[50]  A. Tversky,et al.  The framing of decisions and the psychology of choice. , 1981, Science.

[51]  Michael I. Norton,et al.  Psychology and Experimental Economics , 2007 .

[52]  Gary E. Bolton,et al.  A Laboratory Study of the Benefits of Including Uncertainty Information in Weather Forecasts , 2006 .

[53]  A. H. Murphy,et al.  Misinterpretations of precipitation probability forecasts , 1980 .

[54]  V. Smith Monetary Rewards and Decision Cost in Experimental Economics: An Extension , 1993 .

[55]  Jürgen Schupp,et al.  Measuring Trust: Experiments and Surveys in Contrast and Combination , 2009, SSRN Electronic Journal.

[56]  Rebecca E. Morss,et al.  300 Billion Served: Sources, Perceptions, Uses, and Values of Weather Forecasts , 2009 .

[57]  A. H. Murphy,et al.  Forecast value: prototype decision-making models , 1997 .