Comparison of a single-model EPS with a multi-model ensemble consisting of a few operational models

Since the introduction of operational ensemble forecasts in Numerical Weather Prediction(NWP) more than 5 years ago, the dispute on how to best determine the initial perturbationshas largely dominated the direction of research in the field of ensemble prediction. While it isimportant to consider uncertainties in the initial condition, errors due to model physics or themodel numerics and truncation provide another source of forecast errors and might also beconsidered in ensemble prediction. In this study, we compare the performance of 2 fundamentallydifferent ensemble schemes. First, the ensemble prediction system (EPS) of the European Centrefor Medium Range Forecasts is taken as a representative of the single-model approach basedon the perfect model assumption and thus taking only the uncertainty in the observations intoaccount. Second, a virtual ensemble comprised of the operational forecasts of 4 NWP centersas a “gratis” candidate of the multi-model approach which, in addition, takes model errors intoaccount. The comparison is based on forecasts of 500 hPa fields over Europe for a summer anda winter period in 1997 and on diagnostics ranging from various measures for the performanceof the ensemble means to the statistical consistency and discrimination properties of theensembles. The different sizes of both ensembles poses the main difficulty for the interpretationof the results. If the ensemble size is not considered as a criterion for the evaluation, the resultslead to controversial conclusions; but when penalizing for an overly large and inefficientensemble the results are for the most part consistent, and one has to conclude that the multimodelensemble performs better in most forecast aspects.

[1]  B. M. Lewis,et al.  OBJECTIVE PREDICTION OF FIVE-DAY MEAN TEMPERATURES DURING WINTER , 1959 .

[2]  E. Lorenz.  Deterministic nonperiodic flow , 1963 .

[3]  C. Leith.  Theoretical Skill of Monte Carlo Forecasts , 1974 .

[4]  E. Pitcher.  Application of Stochastic Dynamic Prediction to Real Data , 1977 .

[5]  R. P. Pearce,et al.  Large-Scale Dynamical Processes in the Atmosphere , 1983 .

[6]  A. H. Murphy,et al.  Probability, Statistics, And Decision Making In The Atmospheric Sciences , 1985 .

[7]  D. Ruelle,et al.  Ergodic theory of chaos and strange attractors , 1985 .

[8]  Yoshikazu Hayashi,et al.  Statistical Interpretations of Ensemble-Time Mean Predictability , 1986 .

[9]  Timothy W. Barker,et al.  The Relationship between Spread and Forecast Error in Extended-range Forecasts , 1991 .

[10]  M. J. P. Cullen,et al.  The unified forecast/climate model , 1993 .

[11]  Eugenia Kalnay,et al.  Ensemble Forecasting at NMC: The Generation of Perturbations , 1993 .

[12]  Franco Molteni,et al.  Predictability and finite‐time instability of the northern winter circulation , 1993 .

[13]  A. H. Murphy,et al.  What Is a Good Forecast? An Essay on the Nature of Goodness in Weather Forecasting , 1993 .

[14]  Roberto Buizza,et al.  The Singular-Vector Structure of the Atmospheric Global Circulation , 1995 .

[15]  David Draper,et al.  Assessment and Propagation of Model Uncertainty , 2011 .

[16]  Daniel S. Wilks,et al.  Statistical Methods in the Atmospheric Sciences: An Introduction , 1995 .

[17]  F. Molteni,et al.  The ECMWF Ensemble Prediction System: Methodology and validation , 1996 .

[18]  Philippe Courtier,et al.  Sensitivity of forecast errors to initial conditions , 1996 .

[19]  Jeffrey L. Anderson.  A Method for Producing and Evaluating Probabilistic Forecasts from Ensemble Model Integrations , 1996 .

[20]  P. L. Houtekamer,et al.  A System Simulation Approach to Ensemble Prediction , 1996 .

[21]  Leonard A. Smith,et al.  Accountability and internal consistency in ensemble formation , 1997 .

[22]  Roberto Buizza,et al.  The ECMWF Ensemble Prediction System , 1997 .

[23]  Zoltan Toth,et al.  A Synoptic Evaluation of the NCEP Ensemble , 1997 .

[24]  Zoltan Toth,et al.  An Ensemble Forecasting Primer , 1997 .

[25]  M. Ehrendorfer.  Vorhersage der Unsicherheit numerischer Wetterprognosen: eine Übersicht , 1997 .

[26]  T. Hamill,et al.  Evaluation of Eta-RSM Ensemble Probabilistic Precipitation Forecasts , 1998 .

[27]  William R. Burrows,et al.  A Strategy for Verification of Weather Element Forecasts from an Ensemble Prediction System , 1999 .

[28]  F. Atger,et al.  The Skill of Ensemble Prediction Systems , 1999 .