Skill assessment of seasonal hindcasts from the Canadian historical forecast project

Abstract The performance of seasonal hindcasts produced with four global atmospheric models in the second phase of the Canadian Historical Forecasting Project is evaluated. Deterministic and probabilistic forecast skill assessments are carried out using common verification measures. Several methods of combining multi‐model output to produce deterministic and probabilistic forecasts of near‐surface air temperature, 500 hPa geopotential height, and 700 hPa temperature for zero‐month and one‐month leads are considered. A variance‐based weighting modestly improves the skill of deterministic and probabilistic hindcasts in some cases. A parametric Gaussian probability estimator is superior to a non‐parametric count‐method estimator for producing multi‐model probability forecasts. Statistical adjustment is beneficial for deterministic and probabilistic hindcasts of near‐surface temperature over the ocean but not always over land. Skill improves with the number of different models used for a given total ensemble size. The four‐model ensemble is shown to be a reasonable multi‐model configuration.

[1]  J. Swets.  The Relative Operating Characteristic in Psychology , 1973, Science.

[2]  J. Shukla,et al.  Current status of ENSO prediction skill in coupled ocean–atmosphere models , 2008 .

[3]  A. H. Murphy.  A New Vector Partition of the Probability Score , 1973 .

[4]  David Hinkley,et al.  Bootstrap Methods: Another Look at the Jackknife , 2008 .

[5]  Xuebin Zhang,et al.  Temperature and precipitation trends in Canada during the 20th century , 2000, Data, Models and Analysis.

[6]  F. Zwiers,et al.  Climate Predictions with Multimodel Ensembles , 2002 .

[7]  Renate Hagedorn,et al.  The rationale behind the success of multi-model ensembles in seasonal forecasting — I. Basic concept , 2005 .

[8]  John M. Wallace,et al.  Planetary-Scale Atmospheric Phenomena Associated with the Southern Oscillation , 1981 .

[9]  H. Volger,et al.  WMO – World Meteorological Organization , 2000, A Concise Encyclopedia of the United Nations.

[10]  A. Sterl,et al.  The ERA‐40 re‐analysis , 2005 .

[11]  R. Reynolds,et al.  The NCEP/NCAR 40-Year Reanalysis Project , 1996, Renewable Energy.

[12]  David A. Plummer,et al.  The CCCma third generation AGCM and its extension into the middle atmosphere , 2008 .

[13]  J. Blanchet,et al.  The Canadian Climate Centre Second-Generation General Circulation Model and Its Equilibrium Climate , 1992 .

[14]  Tim Palmer,et al.  The prospects for seasonal forecasting—A review paper , 1994 .

[15]  A. Barnston,et al.  A Degeneracy in Cross-Validated Skill in Regression-based Forecasts , 1993 .

[16]  Initialization and Predictability of a Coupled ENSO Forecast Model , 1997 .

[17]  Leonard A. Smith,et al.  Uncertainty in predictions of the climate response to rising levels of greenhouse gases , 2005, Nature.

[18]  Lucie A. Vincent,et al.  Canadian historical and homogeneous temperature datasets for climate change analyses , 1999 .

[19]  G. Brier.  VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY , 1950 .

[20]  L. Isaksen,et al.  The ERA-40 Reanalysis , 2004 .

[21]  Andrew P. Morse,et al.  DEVELOPMENT OF A EUROPEAN MULTIMODEL ENSEMBLE SYSTEM FOR SEASONAL-TO-INTERANNUAL PREDICTION (DEMETER) , 2004 .

[22]  I. Jolliffe,et al.  Forecast verification : a practitioner's guide in atmospheric science , 2011 .

[23]  Francis W. Zwiers,et al.  Seasonal predictions based on two dynamical models , 2001 .

[24]  Francis W. Zwiers,et al.  Improved Seasonal Probability Forecasts , 2003 .

[25]  A. Staniforth,et al.  The Operational CMC–MRB Global Environmental Multiscale (GEM) Model. Part I: Design Considerations and Formulation , 1998 .

[26]  P. L. Houtekamer,et al.  A System Simulation Approach to Ensemble Prediction , 1996 .

[27]  Tim N. Palmer,et al.  Seasonal skill and predictability of ECMWF PROVOST ensembles , 2000 .

[28]  G. Boer,et al.  CMIP1 evaluation and intercomparison of coupled climate models , 2001 .

[29]  F. Zwiers,et al.  Skill of seasonal hindcasts as a function of the ensemble size , 2001 .

[30]  W. Briggs.  Statistical Methods in the Atmospheric Sciences , 2007 .

[31]  Renate Hagedorn,et al.  The rationale behind the success of multi-model ensembles in seasonal forecasting-II , 2005 .

[32]  Peter J. Webster,et al.  Random error growth in NMC's global forecasts , 1994 .

[33]  F. Molteni,et al.  The ECMWF Ensemble Prediction System: Methodology and validation , 1996 .

[34]  Bin Wang,et al.  Advance and prospectus of seasonal prediction: assessment of the APCC/CliPAS 14-model ensemble retrospective seasonal prediction (1980–2004) , 2009 .

[35]  David A. Plummer,et al.  Technical Note: The CCCma third generation AGCM and its extension into the middle atmosphere , 2008 .

[36]  J. Horel,et al.  Observed and Simulated Intraseasonal Variability of the Wintertime Planetary Circulation , 1988 .

[37]  A. Barnston,et al.  Dominant Factors Influencing the Seasonal Predictability of U.S. Precipitation and Surface Air Temperature , 2000 .

[38]  Eugenia Kalnay,et al.  Operational Ensemble Prediction at the National Meteorological Center: Practical Aspects , 1993 .

[39]  Huug van den Dool,et al.  An analysis of multimodel ensemble predictions for seasonal climate anomalies , 2002 .

[40]  A. H. Murphy,et al.  The attributes diagram A geometrical framework for assessing the quality of probability forecasts , 1986 .

[41]  Harold Ritchie.  Application of the semi-Lagrangian method to a multilevel spectral primitive-equations model , 1991 .