Ensemble predictions and perceptions of risk, uncertainty, and error in flood forecasting

Abstract Under the auspices of the World Meteorological Organization, there are a number of international initiatives to promote the development and use of so-called ensemble prediction systems (EPS) for flood forecasting. The campaign to apply these meteorological techniques to flood forecasting raises important questions about how the probabilistic information these systems provide can be used for what in operational terms is typically a binary decision of whether or not to issue a flood warning. To explore these issues, we report on the results of a series of focus group discussions conducted with operational flood forecasters from across Europe on behalf of the European Flood Alert System. Working in small groups to simulate operational conditions, forecasters engaged in a series of carefully designed forecasting exercises using various different combinations of actual data from real events. Focus group data was supplemented by a follow-up questionnaire survey exploring how flood forecasters understand risk, uncertainty, and error. Results suggest that flood forecasters may not instinctively use ensemble predictions in the way that promoters of EPS perhaps think they should. The paper concludes by exploring the implications of these divergent ‘epistemic cultures’ for efforts to apply ensemble prediction techniques developed in the context of weather forecasting to the rather different one of flood forecasting.

[1]  B. Flyvbjerg,et al.  Making Social Science Matter: Why Social Inquiry Fails and How It Can Succeed Again , 2001 .

[2]  Rebecca E. Morss,et al.  Flood Risk, Uncertainty, and Scientific Information for Decision Making: Lessons from an Interdisciplinary Project , 2005 .

[3]  Adrian E. Raftery,et al.  Weather Forecasting with Ensemble Methods , 2005, Science.

[4]  A. Hollingsworth,et al.  Probabilistic Predictions of Precipitation Using the ECMWF Ensemble Prediction System , 1999 .

[5]  Keith Beven,et al.  A manifesto for the equifinality thesis , 2006 .

[6]  Simon Shackley,et al.  INTEGRATING KNOWLEDGES FOR CLIMATE-CHANGE - PYRAMIDS, NETS AND UNCERTAINTIES , 1995 .

[7]  B. Flyvbjerg,et al.  Making Social Science Matter: Why Social Inquiry Fails and How It Can Succeed Again , 2001 .

[8]  Tim N. Palmer,et al.  The economic value of ensemble forecasts as a tool for risk assessment: From days to decades , 2002 .

[9]  C. Leith Theoretical Skill of Monte Carlo Forecasts , 1974 .

[10]  Leonard A. Smith,et al.  Using ensemble weather forecasts to manage utilities risk , 2004 .

[11]  F. Pappenberger,et al.  Ignorance is bliss: Or seven reasons not to use uncertainty analysis , 2006 .

[12]  S. Clegg Making Social Science Matter: Why Social Inquiry Fails and How It Can Succeed Again , 2002 .

[13]  W. Edwards Behavioral decision theory. , 1961, Annual review of psychology.

[14]  George Gaskell,et al.  A theory of risk colonization: The spiralling regulatory logics of societal and institutional risk , 2006 .

[15]  C. P. Goodman,et al.  The Tacit Dimension , 2003 .

[16]  Virginia Murray The Precautionary Principle in the 20th Century—Late Lessons from Early Warnings , 2004 .

[17]  Thomas R. Stewart,et al.  Uncertainty, Judgment, and Error in Prediction , 2000 .

[18]  A. H. Murphy,et al.  Misinterpretations of precipitation probability forecasts , 1980 .

[19]  M. Ehrendorfer Vorhersage der Unsicherheit numerischer Wetterprognosen: eine Übersicht , 1997 .

[20]  John Handmer,et al.  The Role of Unofficial Flood Warning Systems , 1998 .

[21]  D. Kahneman,et al.  Heuristics and Biases: The Psychology of Intuitive Judgment , 2002 .

[22]  David Demeritt,et al.  The Construction of Global Warming and the Politics of Science , 2001 .

[23]  Gerald van der Grijn,et al.  Predictability of Weather and Climate: Ensemble forecasts: can they provide useful early warnings? , 2006 .

[24]  T. R. Stewart,et al.  Effects of improved information on the components of skill in weather forecasting , 1992 .

[25]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[26]  T. P. Legg,et al.  Early Warnings of Severe Weather from Ensemble Forecast Information , 2004 .

[27]  Ecmwf Newsletter,et al.  EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS , 2004 .

[28]  Ramón de Elía,et al.  Diversity in Interpretations of Probability: Implications for Weather Forecasting , 2005 .

[29]  Newsha K. Ajami,et al.  Hydrologic ensemble prediction experiment focuses on reliable forecasts , 2005 .

[30]  K. Hymas The Meteorological Office National Severe Weather Warning Service (NSWWS) , 1993 .

[31]  R. Pielke,et al.  Who decides? Forecasts and responsibilities in the 1997 Red River flood , 1999 .

[32]  Kenneth R. Hammond,et al.  Human Judgment and Social Policy: Irreducible Uncertainty, Inevitable Error, Unavoidable Injustice , 2000 .

[33]  S. Rayner,et al.  Weather Forecasts are for Wimps: Why Water Resource Managers Do Not Use Climate Forecasts , 2005 .

[34]  R. Pielke,et al.  Prediction : science, decision making, and the future of nature , 2000 .

[35]  K. Knorr-Cetina,et al.  Epistemic cultures : how the sciences make knowledge , 1999 .

[36]  R. Buizza,et al.  A Comparison of the ECMWF, MSC, and NCEP Global Ensemble Prediction Systems , 2005 .

[37]  Neville Nicholls,et al.  Cognitive Illusions, Heuristics, and Climate Prediction , 1999 .

[38]  David H. DeVorkin,et al.  Inventing Accuracy: A Historical Sociology of Nuclear Missile Guidance , 1990 .

[39]  R. Buizza,et al.  Using weather ensemble predictions in electricity demand forecasting , 2003 .

[40]  Charles A. Doswell,et al.  Weather Forecasting by Humans—Heuristics and Decision Making , 2004 .