RELM Testing Center

We describe the organizational setup and the set of rules developed to test earthquake likelihood models of the Regional Earthquake Likelihood Models (RELM) initiative. Truly prospective testing for a multitude of forecasts in an unbiased and reproducible way requires a set of rules to be obeyed by each model as well as a cyber-infrastructure: the testing center. These rules encompass free parameters, e.g., the testing area and grid, forecast periods, magnitude ranges, earthquake catalogs provided to the models and used for testing, and declustering. A combination of these free parameters defines a class of models. Within RELM we distinguish between five-year models, one-year models, and one-day models, which issue their forecast for the respective periods. While five-year models provide their forecasts as numbers, one-year and one-day models must be installed in the testing center together with their source code to enable yearly or daily forecast generation. Only with installed models can results be reproduced at any time. The declared RELM goal of testing a multitude of forecast models in a prospective (forward-looking) sense is, to our knowledge, unique in seismology. It is the first time that a group of modelers agree to submit their models to a common, community-agreed test against future seismicity. We believe that this is an important milestone for forecasting- and prediction-related research in seismology because it offers an opportunity to overcome past shortcomings and deadlocks. The primary goal of the common testing is not to find the ultimate winner but to advance our current physical understanding and/or the statistical description of the way earthquakes occur. Ultimately this will lead in small steps (or, possibly, giant leaps) to better forecast models. Prospective testing of models in seismology is rare but a few laudable examples exist (Kagan and Jackson 1995, Evison and Rhoades 1999, Rong et …