Reliability and effectiveness of early warning systems for natural hazards: Concept and application to debris flow warning

Early Warning Systems (EWS) are increasingly applied to mitigate the risks posed by natural hazards. To compare the effect of EWS with alternative risk reduction measures and to optimize their design and operation, their reliability and effectiveness must be quantified. In the present contribution, a framework approach to the evaluation of threshold-based EWS for natural hazards is presented. The system reliability is classically represented by the Probability of Detection (POD) and Probability of False Alarms (PFA). We demonstrate how the EWS effectiveness, which is a measure of risk reduction, can be formulated as a function of POD and PFA. To model the EWS and compute the reliability, we develop a framework based on Bayesian Networks, which is further extended to a decision graph, facilitating the optimization of the warning system. In a case study, the framework is applied to the assessment of an existing debris flow EWS. The application demonstrates the potential of the framework for identifying the important factors influencing the effectiveness of the EWS and determining optimal warning strategies and system configurations.

[1]  Finn V. Jensen,et al.  Bayesian Networks and Decision Graphs , 2001, Statistics for Engineering and Information Science.

[2]  R. A. Sturny,et al.  Bayesian Networks for Assessing the Reliability of a Glacier Lake Warning System in Switzerland , 2014 .

[3]  Daniel Straub,et al.  Classification of warning systems for natural hazards , 2012 .

[4]  Francesco Napolitano,et al.  Rainfall thresholds and flood warning: an operative case study , 2009 .

[5]  Doris Hattenberger,et al.  (Naturgefahren-)Mess- und Frühwarnsysteme: Einzelne rechtliche Aspekte , 2008 .

[6]  Michael Havbro Faber,et al.  Quantitative risk-cost-benefit analysis of selected mitigation options for two case studies , 2012 .

[7]  J. Chatterton,et al.  The Benefits of Flood and Coastal Risk Management: A Handbook of Assessment Techniques , 2005 .

[8]  Christoph M. Rheinberger.  Learning from the past: statistical performance measures for avalanche warning services , 2012, Natural Hazards.

[9]  Ross D. Shachter.  Evaluating Influence Diagrams , 1986, Oper. Res..

[10]  Michael Bründl,et al.  The risk concept and its application in natural hazard risk management in Switzerland , 2009 .

[11]  Adrienne Grêt-Regamey,et al.  Spatially explicit avalanche risk assessment linking Bayesian networks to a GIS , 2006 .

[12]  Daniel Straub,et al.  Natural hazards risk assessment using Bayesian networks , 2005 .

[13]  Daniel Straub,et al.  Forecasting rock slope failure: how reliable and effective are warning systems? , 2016, Landslides.

[14]  Rafael Rumí,et al.  Bayesian networks in environmental modelling , 2011, Environ. Model. Softw..

[15]  C. Rubin,et al.  Effectiveness and Efficiency of Early Warning Systems for Flash Floods , 2007 .

[16]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[17]  M E Paté-Cornell,et al.  Warning systems in risk management. , 1986, Risk analysis : an official publication of the Society for Risk Analysis.

[18]  Daniel Straub,et al.  Bayesian networks to quantify the reliability of a debris flow alarm system , 2013 .

[19]  David T. Ford,et al.  QUANTIFYING THE BENEFIT OF A FLOOD WARNING SYSTEM , 2004 .

[20]  Farrokh Nadim,et al.  Stochastic design of an early warning system , 2008 .

[21]  Luigi Portinale,et al.  Bayesian networks in reliability , 2007, Reliab. Eng. Syst. Saf..

[22]  M. Bründl,et al.  Reliability analysis of the Swiss avalanche warning system , 2011 .

[23]  Kevin M. Simmons,et al.  False Alarms, Tornado Warnings, and Tornado Casualties , 2009 .

[24]  John A. Swets,et al.  Signal Detection Theory and ROC Analysis in Psychology and Diagnostics: Collected Papers , 1996 .

[25]  S. Breznitz.  Cry Wolf: The Psychology of False Alarms , 1984 .

[26]  M. Zweig,et al.  Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. , 1993, Clinical chemistry.

[27]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[28]  Michael Havbro Faber,et al.  Proceedings of the 8th International Conference on Structural Safety and Reliability , 2002 .

[29]  David P. Rogers,et al.  Global assessment report on disaster risk reduction : costs and benefits of early warning systems , 2010 .

[30]  Franck Schoefs,et al.  Assessment of ROC curves for inspection of random fields , 2009 .

[31]  W. E. Vesely,et al.  Two measures of risk importance and their application , 1985 .

[32]  Jayanta K. Ghosh,et al.  Bayesian Networks and Decision Graphs, 2nd Edition by Finn V. Jensen, Thomas D. Nielsen , 2008 .

[33]  Michael S. Wogalter.  POSTEXPOSURE EVALUATION OF WARNING EFFECTIVENESS: A REVIEW OF FIELD STUDIES AND POPULATION-BASED RESEARCH , 2006 .

[34]  Clifton A. Ericson,et al.  Hazard Analysis Techniques for System Safety , 2005 .

[35]  Dou Long,et al.  Reliability of Flood Warning Systems , 1994 .

[36]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[37]  Aarnout Brombacher,et al.  New quantitative safety standards : different techniques, different results? , 1998 .

[38]  Tova Rosenbloom,et al.  Crossing at a red light: Behaviour of individuals and groups , 2009 .

[39]  J P Bliss,et al.  Human probability matching behaviour in response to alarms of varying reliability. , 1995, Ergonomics.

[40]  W. W. Peterson,et al.  The theory of signal detectability , 1954, Trans. IRE Prof. Group Inf. Theory.

[41]  Florian Pappenberger,et al.  Ensemble hydro-meteorological simulation for flash flood early detection in southern Switzerland , 2012 .

[42]  Stuart Newstead,et al.  Riding through red lights: the rate, characteristics and risk factors of non-compliant urban commuter cyclists. , 2011, Accident; analysis and prevention.

[43]  Stefan Margreth,et al.  Effectiveness of mitigation measures against natural hazards , 2010 .

[44]  Frank Scherbaum,et al.  Bayesian networks for tsunami early warning , 2011 .

[45]  R. Pöttler,et al.  Georisk: Assessment and Management of Risk for Engineered Systems and Geohazards , 2007 .

[46]  Daniel Straub,et al.  Bayesian Network Enhanced with Structural Reliability Methods: Methodology , 2010, 1203.5986.

[47]  Jakob Rhyner,et al.  A debris-flow alarm system for the Alpine Illgraben catchment: design and performance , 2009 .