How sensitive are probabilistic precipitation forecasts to the choice of calibration algorithms and the ensemble generation method? Part I: sensitivity to calibration methods

Different techniques for obtaining probabilistic quantitative precipitation forecasts (PQPFs) over South America are tested during the 2002–2003 warm season. They have been applied to a regional ensemble system which uses the breeding technique to generate initial and boundary conditions perturbations. This comparison involves seven algorithms and also includes experiments to select an adequate size for the training period. Results show that the sensitivity to different calibration strategies is small with the exception of the rank histogram algorithm. The inclusion of the ensemble spread or the use of different ensemble members for the computation of probabilities shows almost no improvement with respect to probabilistic forecasts computed using the ensemble mean. This is basically due to the strong relationship between precipitation error and its amount. Copyright © 2011 Royal Meteorological Society

[1]  G. Brier.  VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY , 1950 .

[2]  E. Epstein,et al.  Stochastic dynamic prediction , 1969 .

[3]  Rex J. Fleming,et al.  ON STOCHASTIC DYNAMIC PREDICTION , 1971 .

[4]  A. Betts.  A new convective adjustment scheme. Part I: Observational and theoretical basis , 1986 .

[5]  J. Schaefer.  The critical success index as an indicator of Warning skill , 1990 .

[6]  Eugenia Kalnay,et al.  Ensemble Forecasting at NMC: The Generation of Perturbations , 1993 .

[7]  Z. Janjic.  The Step-Mountain Eta Coordinate Model: Further Developments of the Convection, Viscous Sublayer, and Turbulence Closure Schemes , 1994 .

[8]  Daniel S. Wilks,et al.  Statistical Methods in the Atmospheric Sciences: An Introduction , 1995 .

[9]  F. Molteni,et al.  The ECMWF Ensemble Prediction System: Methodology and validation , 1996 .

[10]  H. Pan,et al.  Nonlocal Boundary Layer Vertical Diffusion in a Medium-Range Forecast Model , 1996 .

[11]  Stanley G. Benjamin,et al.  Performance of Different Soil Model Configurations in Simulating Ground Surface Temperature and Surface Fluxes , 1997 .

[12]  E. Kalnay,et al.  Ensemble Forecasting at NCEP and the Breeding Method , 1997 .

[13]  Thomas M. Hamill,et al.  Verification of Eta–RSM Short-Range Ensemble Forecasts , 1997 .

[14]  T. Hamill,et al.  Evaluation of Eta-RSM Ensemble Probabilistic Precipitation Forecasts , 1998 .

[15]  T. N. Krishnamurti,et al.  Improved Weather and Seasonal Climate Forecasts from Multimodel Superensemble. , 1999, Science.

[16]  J. Dudhia,et al.  Coupling an Advanced Land Surface–Hydrology Model with the Penn State–NCAR MM5 Modeling System. Part I: Model Implementation and Sensitivity , 2001 .

[17]  D. Culler,et al.  Comparison of methods , 2000 .

[18]  H. Hersbach.  Decomposition of the Continuous Ranked Probability Score for Ensemble Prediction Systems , 2000 .

[19]  David S. Richardson,et al.  ON THE ECONOMIC VALUE OF ENSEMBLE BASED WEATHER FORECASTS , 2001 .

[20]  T. Hamill.  Interpretation of Rank Histograms for Verifying Ensemble Forecasts , 2001 .

[21]  K. Droegemeier,et al.  Objective Verification of the SAMEX ’98 Ensemble Forecasts , 2001 .

[22]  E. Ebert.  Ability of a Poor Man's Ensemble to Predict the Probability and Distribution of Precipitation , 2001 .

[23]  Richard L. Pfeffer,et al.  Comparison of Methodologies for Probabilistic Quantitative Precipitation Forecasting , 2002 .

[24]  G. Grell,et al.  A generalized approach to parameterizing convection combining ensemble and data assimilation techniques , 2002 .

[25]  M. Baldwin,et al.  Supplement to The WGNE Assessment of Short-term Quantitative Precipitation Forecasts , 2003 .

[26]  M. Baldwin,et al.  THE WGNE ASSESSMENT OF SHORT-TERM QUANTITATIVE PRECIPITATION FORECASTS , 2003 .

[27]  Improved Results for Probabilistic Quantitative Precipitation Forecasting , 2003 .

[28]  Thomas M. Hamill,et al.  Ensemble Reforecasting: Improving Medium-Range Forecast Skill Using Retrospective Forecasts , 2004 .

[29]  J. Janowiak,et al.  CMORPH: A Method that Produces Global Precipitation Estimates from Passive Microwave and Infrared Data at High Spatial and Temporal Resolution , 2004 .

[30]  R. Houze.  Mesoscale convective systems , 2004 .

[31]  Moti Segal,et al.  Does Increased Predicted Warm-Season Rainfall Indicate Enhanced Likelihood of Rain Occurrence? , 2004 .

[32]  John S. Kain,et al.  The Kain–Fritsch Convective Parameterization: An Update , 2004 .

[33]  A. Raftery,et al.  Using Bayesian Model Averaging to Calibrate Forecast Ensembles , 2005 .

[34]  Anton H. Westveld,et al.  Calibrated Probabilistic Forecasting Using Ensemble Model Output Statistics and Minimum CRPS Estimation , 2005 .

[35]  D. Wilks.  Effects of stochastic parametrizations in the Lorenz '96 system , 2005 .

[36]  U. Damrath,et al.  Probabilistic precipitation forecasts from a deterministic model: a pragmatic approach , 2005 .

[37]  Jordan G. Powers,et al.  A Description of the Advanced Research WRF Version 2 , 2005 .

[38]  John E. Janowiak,et al.  Diurnal cycle of precipitation determined from the CMORPH high spatial and temporal resolution global precipitation analyses , 2005 .

[39]  Thomas M. Hamill,et al.  Probabilistic Quantitative Precipitation Forecasts Based on Reforecast Analogs: Theory and Application , 2006 .

[40]  Thomas M. Hamill,et al.  Comparison of Ensemble-MOS Methods Using GFS Reforecasts , 2007 .

[41]  Florian Pappenberger,et al.  The Ensemble Prediction System - Recent and Ongoing Developments , 2007 .

[42]  W. Gallus,et al.  Evaluation of probabilistic precipitation forecasts determined from Eta and AVN forecasted amounts , 2007 .

[43]  Edward J. Zipser,et al.  Mesoscale Convective Systems over Southeastern South America and Their Relationship with the South American Low-Level Jet , 2007 .

[44]  J. Janowiak,et al.  COMPARISON OF NEAR-REAL-TIME PRECIPITATION ESTIMATES FROM SATELLITE OBSERVATIONS AND NUMERICAL MODELS , 2007 .

[45]  J. M. Sloughter,et al.  Probabilistic Quantitative Precipitation Forecasting Using Bayesian Model Averaging , 2007 .

[46]  D. Stensrud,et al.  Reliable Probabilistic Quantitative Precipitation Forecasts from a Short-Range Ensemble Forecasting System , 2007 .

[47]  I. Jolliffe,et al.  Two Extra Components in the Brier Score Decomposition , 2008 .

[48]  Roberto Buizza,et al.  TIGGE: Preliminary results on comparing and combining ensembles , 2008 .

[49]  Daniel S. Wilks,et al.  Extending logistic regression to provide full‐probability‐distribution MOS forecasts , 2009 .

[50]  E. Kalnay,et al.  Comparison of Methods Used to Generate Probabilistic Quantitative Precipitation Forecasts over South America , 2009 .

[51]  Juan Ruiz,et al.  Evaluation of different methodologies for precipitation estimates calibration - cmorph - over South America , 2009 .

[52]  M. Schmeits,et al.  A Comparison between Raw Ensemble Output, (Modified) Bayesian Model Averaging, and Extended Logistic Regression Using ECMWF Ensemble Precipitation Reforecasts , 2010 .

[53]  Celeste Saulo,et al.  WRF Model Sensitivity to Choice of Parameterization over South America: Validation against Surface Variables , 2010 .

[54]  A. Raftery,et al.  Calibrating Multimodel Forecast Ensembles with Exchangeable and Missing Members Using Bayesian Model Averaging , 2010 .

[55]  Moti Segal,et al.  Improving probabilistic ensemble forecasts of convection through the application of QPF-POP relationships , 2011 .

[56]  E. Kalnay,et al.  How sensitive are probabilistic precipitation forecasts to the choice of calibration algorithms and the ensemble generation method? Part II: sensitivity to ensemble generation method , 2012 .

[57]  F. Anthony Eckel,et al.  Calibrated Probabilistic Quantitative Precipitation Forecasts Based on the MRF Ensemble , 2012 .