Verification of the ECMWF ensemble forecasts of wind speed against analyses and observations

A framework for the verification of ensemble forecasts of near-surface wind speed is described. It is based on existing scores and diagnostic tools, though considering observations from synoptic stations as reference instead of the analysis. This approach is motivated by the idea of having a user-oriented view of verification, for instance with the wind power applications in mind. The verification framework is specifically applied to the case of ECMWF ensemble forecasts and over Europe. Dynamic climatologies are derived at the various stations, serving as a benchmark. The impact of observational uncertainty on scores and diagnostic tools is also considered. The interest of this framework is demonstrated from its application to the routine evaluation of ensemble forecasts and to the assessment of the quality improvements brought in by the recent change in horizontal resolution of the ECMWF ensemble prediction system. The most important conclusions cover (1) the generally high skill of these ensemble forecasts of near-surface wind speed when evaluated at synoptic stations, (2) the noteworthy improvement of scores brought by the change of horizontal resolution, and, (3) the scope for further improvements of reliability and skill of wind speed ensemble forecasts by appropriate post-processing. Copyright © 2011 Royal Meteorological Society

[1]  A. H. Murphy,et al.  What Is a Good Forecast? An Essay on the Nature of Goodness in Weather Forecasting , 1993 .

[2]  M. Lange,et al.  Physical Approach to Short-Term Wind Power Prediction , 2005 .

[3]  Tilmann Gneiting,et al.  Evaluating Point Forecasts , 2009 .

[4]  A. H. Murphy,et al.  A General Framework for Forecast Verification , 1987 .

[5]  R. Buizza,et al.  The Skill of Probabilistic Precipitation Forecasts under Observational Uncertainties within the Generalized Likelihood Uncertainty Estimation Framework for Hydrological Applications , 2009 .

[6]  Thomas M. Hamill,et al.  Measuring forecast skill: is it real skill or is it the varying climatology? , 2006 .

[7]  Daniel Rousseau,et al.  Economic value of weather and Climate forecasts , 1998 .

[8]  Pierre Pinson,et al.  Non‐parametric probabilistic forecasts of wind power: required properties and evaluation , 2007 .

[9]  Kenneth F. Wallis,et al.  Density Forecasting: A Survey , 2000 .

[10]  Henrik Madsen,et al.  A review on the young history of the wind power short-term prediction , 2008 .

[11]  Martin Leutbecher,et al.  Scale‐dependent verification of ensemble forecasts , 2008 .

[12]  Martin Leutbecher,et al.  Comparison between Singular Vectors and Breeding Vectors as Initial Perturbations for the ECMWF Ensemble Prediction System , 2008 .

[13]  P. J. Green,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[14]  O. Talagrand,et al.  Impact of observational error on the validation of ensemble prediction systems , 2008 .

[15]  Daryn Lehoux,et al.  A mighty wind , 2008, Clinical toxicology.

[16]  R. Buizza,et al.  A Comparison of the ECMWF, MSC, and NCEP Global Ensemble Prediction Systems , 2005 .

[17]  A. H. Murphy,et al.  Economic Value of Weather And Climate Forecasts: Contents , 1997 .

[18]  Tim N. Palmer,et al.  Ensemble forecasting , 2008, J. Comput. Phys..

[19]  Tilmann Gneiting,et al.  Editorial: Probabilistic forecasting , 2008 .

[20]  P. L. Houtekamer,et al.  Verification of an Ensemble Prediction System against Observations , 2007 .

[21]  P. Pinson,et al.  Trading Wind Generation From Short-Term Probabilistic Forecasts of Wind Power , 2007, IEEE Transactions on Power Systems.

[22]  T. Gneiting.  Quantiles as optimal point forecasts , 2011 .

[23]  David M. Bradley,et al.  On the Distribution of the Sum of n Non-Identically Distributed Uniform Random Variables , 2002, math/0411298.

[24]  T. Gneiting.  Quantiles as Optimal Point Predictors , 2008 .

[25]  Renate Hagedorn,et al.  Representing model uncertainty in weather and climate prediction , 2005 .

[26]  H. Madsen,et al.  Reliability diagrams for non‐parametric density forecasts of continuous variables: Accounting for serial correlation , 2010 .

[27]  Simon J. Mason,et al.  Understanding forecast verification statistics , 2008 .

[28]  Samarendra Kumar Mitra.  On the Probability Distribution of the Sum of Uniformly Distributed Random Variables , 1971 .

[29]  M A Matos,et al.  Setting the Operating Reserve Using Probabilistic Wind Power Forecasts , 2011, IEEE Transactions on Power Systems.

[30]  W. Shen,et al.  A simple prediction model of hurricane intensity , 2005 .

[31]  Gregor Giebel,et al.  The State-Of-The-Art in Short-Term Prediction of Wind Power. A Literature Overview , 2003 .

[32]  T. Palmer.  Predicting uncertainty in forecasts of weather and climate , 2000 .

[33]  T. Palmer,et al.  Stochastic representation of model uncertainties in the ECMWF ensemble prediction system , 2007 .

[34]  T. Gneiting.  Making and Evaluating Point Forecasts , 2009, 0912.0902.

[35]  Leonard A. Smith,et al.  Increasing the Reliability of Reliability Diagrams , 2007 .

[36]  Tilmann Gneiting,et al.  Probabilistic forecasts of wind speed: ensemble model output statistics by using heteroscedastic censored regression , 2010 .

[37]  A. Sterl,et al.  The ERA‐40 re‐analysis , 2005 .

[38]  C. Marzban,et al.  On the Effect of Correlations on Rank Histograms: Reliability of Temperature and Wind Speed Forecasts from Finescale Ensemble Reforecasts , 2011 .

[39]  Allan Timmermann,et al.  Density forecasting in economics and finance , 2000 .