A new measure of ensemble performance: Perturbation versus error correlation analysis (PECA)

Abstract Most existing ensemble forecast verification statistics are influenced by the quality of not only the ensemble generation scheme, but also the forecast model and the analysis scheme. In this study, a new tool called perturbation versus error correlation analysis (PECA) is introduced that lessens the influence of the initial errors that affect the quality of the analysis. PECA evaluates the ensemble perturbations, instead of the forecasts themselves, by measuring their ability to explain forecast error variance. As such, PECA offers a more appropriate tool for the comparison of ensembles generated by using different analysis schemes. Ensemble perturbations from both the National Centers for Environmental Prediction (NCEP) and the European Centre for Medium-Range Weather Forecasts (ECMWF) were evaluated and found to perform similarly. The error variance explained by either ensemble increases with the number of members and the lead time. The dynamically conditioned NCEP and ECMWF perturbations outpe...

[1]  J. Holton.  An introduction to dynamic meteorology , 2004 .

[2]  J. Marsden,et al.  Introduction to Dynamics , 1983 .

[3]  John Derber,et al.  The National Meteorological Center's spectral-statistical interpolation analysis system , 1992 .

[4]  Eugenia Kalnay,et al.  Ensemble Forecasting at NMC: The Generation of Perturbations , 1993 .

[5]  H. M. van den Dool,et al.  On the Weights for an Ensemble-Averaged 6–10-Day Forecast , 1994 .

[6]  Roberto Buizza,et al.  The Singular-Vector Structure of the Atmospheric Global Circulation , 1995 .

[7]  F. Molteni,et al.  The ECMWF Ensemble Prediction System: Methodology and validation , 1996 .

[8]  Philippe Courtier,et al.  Sensitivity of forecast errors to initial conditions , 1996 .

[9]  P. L. Houtekamer,et al.  A System Simulation Approach to Ensemble Prediction , 1996 .

[10]  E. Kalnay,et al.  Ensemble Forecasting at NCEP and the Breeding Method , 1997 .

[11]  Istvan Szunyogh,et al.  A comparison of Lyapunov and optimal vectors in a low-resolution GCM , 1997 .

[12]  F. Molteni,et al.  Validation of the ECMWF Ensemble Prediction System Using Empirical Orthogonal Functions , 1999 .

[13]  Ronald M. Errico,et al.  Convergence of Singular Vectors toward Lyapunov Vectors , 1999 .

[14]  F. Atger,et al.  The Skill of Ensemble Prediction Systems , 1999 .

[15]  Roberto Buizza,et al.  3D‐Var Hessian singular vectors and their potential use in the ECMWF ensemble prediction system , 1999 .

[16]  David S. Richardson,et al.  ON THE ECONOMIC VALUE OF ENSEMBLE BASED WEATHER FORECASTS , 2001 .

[17]  David B. Stephenson,et al.  Statistical methods for interpreting Monte Carlo ensemble forecasts , 2000 .

[18]  Suranjana Saha,et al.  Empirical Orthogonal Teleconnections , 2000 .

[19]  D. Richardson.  Skill and relative economic value of the ECMWF ensemble prediction system , 2000 .

[20]  Jorgen S. Frederiksen,et al.  Singular Vectors, Finite-Time Normal Modes, and Error Growth during Blocking , 2000 .

[21]  Mozheng Wei,et al.  Quantifying Local Instability and Predictability of Chaotic Dynamical Systems by Means of Local Metric Entropy , 2000, Int. J. Bifurc. Chaos.

[22]  Craig H. Bishop,et al.  Adaptive sampling with the ensemble transform Kalman filter , 2001 .

[23]  Roberto Buizza,et al.  Tropical singular vectors computed with linearized diabatic physics , 2001 .

[24]  Xuguang Wang,et al.  A Comparison of Breeding and Ensemble Transform Kalman Filter Ensemble Forecast Schemes , 2003 .