Testing alarm‐based earthquake predictions

SUMMARY Motivated by a recent resurgence in earthquake predictability research, we present a method for testing alarm-based earthquake predictions. The testing method is based on the Molchan diagram—a plot of miss rate and fraction of space–time occupied by alarm—and is applicable to a wide class of predictions, including probabilistic earthquake forecasts varying in space, time, and magnitude. A single alarm can be simply tested using the cumulative binomial distribution. Here we consider the more interesting case of a function from which a continuum of well-ordered alarms can be derived. For such an ‘alarm function’ we construct a cumulative performance measure, the area skill score, based on the normalized area above a trajectory on the Molchan diagram. A score of unity indicates perfect skill, a score of zero indicates perfect non-skill, and the expected score for a random alarm function is 1/2. The area skill score quantifies the performance of an arbitrary alarm function relative to a reference model. To illustrate the testing method, we consider the 10-yr experiment by J. Rundle and others to predict M≥ 5 earthquakes in California. We test forecasts from three models: relative intensity (RI), a simple spatial clustering model constructed using only smoothed historical seismicity; pattern informatics (PI), a model that aims to capture seismicity dynamics by pattern recognition; and the U. S. Geological Survey National Seismic Hazard Map (NSHM), a model that comprises smoothed historical seismicity, zones of ‘background’ seismicity, and explicit fault information. Results show that neither PI nor NSHM provide significant performance gain relative to the RI reference model. We suggest that our testing method can be used to evaluate future experiments in the Collaboratory for the Study of Earthquake Predictability and to iteratively improve reference models for earthquake prediction hypothesis testing.

[1]  E. Lehmann.  Testing Statistical Hypotheses , 1960 .

[2]  Y. Kagan,et al.  High-resolution Time-independent Grid-based Forecast for M ≥ 5 Earthquakes in California , 2007 .

[3]  G. M. Molchan,et al.  Earthquake prediction and its optimization , 1992 .

[4]  Alan L. Kafka,et al.  Statistical Analysis of the Hypothesis That Seismicity Delineates Areas Where Future Large Earthquakes Are Likely to Occur in the Central and Eastern United States , 2002 .

[5]  Philip B. Stark,et al.  A few considerations for ascribing statistical significance to earthquake predictions , 1996 .

[6]  John B. Rundle,et al.  Statistical physics approach to understanding the multiscale dynamics of earthquake fault systems , 2003 .

[7]  Walter H. F. Smith,et al.  New, improved version of generic mapping tools released , 1998 .

[8]  G. Molchan,et al.  Structure of optimal strategies in earthquake prediction , 1991 .

[9]  S. Wiemer,et al.  Earthquake Likelihood Model Testing , 2007 .

[10]  A. A. Soloviev,et al.  Nonlinear dynamics of the lithosphere and earthquake prediction , 2003 .

[11]  David A. Rhoades,et al.  Long-range Earthquake Forecasting with Every Earthquake a Precursor According to Scale , 2004 .

[12]  Andrew J. Michael,et al.  Testing prediction methods: Earthquake clustering versus the Poisson Model , 1997 .

[13]  D. Jackson,et al.  Hypothesis testing and earthquake prediction. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[14]  Yan Y. Kagan,et al.  VAN earthquake predictions‒An attempt at statistical evaluation , 1996 .

[15]  J. B. Rundle,et al.  Self-organization in leaky threshold systems: The influence of near-mean field dynamics and its implications for earthquakes, neurobiology, and forecasting , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[16]  Thomas H. Jordan,et al.  Foreshock sequences and short-term earthquake predictability on East Pacific Rise transform faults , 2004, Nature.

[17]  D. Turcotte,et al.  Reverse tracing of short-term earthquake precursors , 2004 .

[18]  Nitin Gupta,et al.  Hazard Calculations for the WGCEP-2002 Earthquake Forecast Using OpenSHA and Distributed Object Technologies , 2005 .

[19]  T. Jordan.  Earthquake Predictability, Brick by Brick , 2006 .

[20]  Stephen E. Fienberg,et al.  Testing Statistical Hypotheses , 2005 .

[21]  Mark D. Petersen,et al.  Time-independent and Time-dependent Seismic Hazard Assessment for the State of California: Uniform California Earthquake Rupture Forecast Model 1.0 , 2007 .

[22]  Edward H. Field,et al.  Overview of the Working Group for the Development of Regional Earthquake Likelihood Models (RELM) , 2005 .

[23]  Ilya Zaliapin,et al.  Short-term earthquake prediction by reverse analysis of lithosphere dynamics , 2006 .

[25]  D. A. Rhoades,et al.  Time-variable factors in earthquake hazard , 1989 .

[26]  Vladimir I. Keilis-borok,et al.  Earthquake Prediction: State-of-the-Art and Emerging Possibilities , 2002 .

[27]  G. M. Molchan,et al.  Strategies in strong earthquake prediction , 1990 .

[28]  J. Douglas Zechar,et al.  The Area Skill Score Statistic for Evaluating Earthquake Predictability Experiments , 2010 .

[29]  Philip B. Stark,et al.  Earthquake prediction: the null hypothesis , 1997 .

[30]  Vladimir Kossobokov,et al.  Premonitory activation of earthquake flow: algorithm M8 , 1990 .

[31]  D. Perkins,et al.  National Seismic-Hazard Maps: Documentation June 1996 , 1996 .

[32]  S. Harmsen,et al.  Documentation for the 2002 update of the national seismic hazard maps , 2002 .

[33]  Kristy F. Tiampo,et al.  Mean-field threshold systems and phase dynamics: An application to earthquake fault systems , 2002 .

[34]  Robert Shcherbakov,et al.  Nonlinear Dynamics of the Lithosphere and Earthquake Prediction , 2003 .

[35]  Vladimir Kossobokov,et al.  Earthquake prediction: basics, achievements, perspectives , 2004 .

[36]  Maximilian J. Werner,et al.  Magnitude uncertainties impact seismic rate estimates, forecasts, and predictability experiments , 2007, 0710.4196.