Methodological aspects of the validation of decadal predictions

Validation techniques based on past performance are well-established in seasonal forecasting. So far there is no consensus on the degree to which these are applicable to decadal predictions. We contribute to this discussion by assessing the effects of drift-correction, cross-val- idation and de-trending. The study employs decadal hindcasts of 2 m temperature from the EU FP6 ENSEMBLES project database and a synthetic toy model. Decadal predictions can be subject to substantial lead-time dependent model drifts. The conventional drift-correction method has a considerable sampling uncertainty, amounting to up to 40% of the potentially predictable signal. Introducing a smooth drift curve allows this uncertainty to be reduced by about 30% for annual values. For drift-corrected decadal predictions the leave-one-out cross-validation procedure may lead to biased skill estimates for decadal prediction due to the small number of hindcasts avail- able. We identify this effect and show that 'jackknifing' represents a suitable technique for esti- mating potential skill without bias and to estimate sampling uncertainty. Results indicate signifi- cant correlation skill on the order of 0.7 to 0.9 for predicting global annual mean temperature on all lead-times. If linear trends are removed prior to verification, skill is still clearly above 0 in the first year for global mean temperature. On a local scale, some specific regions exhibit skill addi- tional to the trend even at longer lead times. With the limited dataset analyzed here, the strong sampling uncertainty still prohibits drawing a final conclusion, by means of verification, on whether or not decadal predictions have skill in predicting climate variability beyond the trend.

[1]  Antje Weisheimer,et al.  Decadal climate prediction with the European Centre for Medium‐Range Weather Forecasts coupled forecast system: Impact of ocean observations , 2011 .

[2]  Francisco J. Doblas-Reyes,et al.  Realistic greenhouse gas forcing and seasonal forecasts , 2007 .

[3]  Shaoqing Zhang,et al.  Impact of observation‐optimized model parameters on decadal predictions: Simulation with a simple pycnocline prediction model , 2011 .

[4]  P. Webster,et al.  Evaluation of short-term climate change prediction in multi-model CMIP 5 decadal hindcasts , 2012 .

[5]  A. Sterl,et al.  The ERA‐40 re‐analysis , 2005 .

[6]  Mark A. Liniger,et al.  Probabilistic Verification of Monthly Temperature Forecasts , 2008 .

[7]  Peter J. Gleckler,et al.  Improved estimates of upper-ocean warming and multi-decadal sea-level rise , 2008, Nature.

[8]  H. L. Miller,et al.  Climate Change 2007: The Physical Science Basis , 2007 .

[9]  Stephen Cusack,et al.  Improved Surface Temperature Prediction for the Coming Decade from a Global Climate Model , 2007, Science.

[10]  Mark A. Liniger,et al.  The discrete brier and ranked probability skill scores , 2007 .

[11]  J. Fyfe,et al.  Skillful predictions of decadal trends in global mean surface temperature , 2011 .

[12]  Mark A. Liniger,et al.  Seasonal Ensemble Forecasts: Are Recalibrated Single Models Better than Multimodels? , 2009 .

[13]  Simona Masina,et al.  Interannual to Decadal Climate Predictability in the North Atlantic: A Multimodel-Ensemble Study , 2006 .

[14]  D. A. Smith,et al.  Dynamics of Decadal Climate Variability and Implications for its Prediction , 2010 .

[15]  A. Barnston,et al.  A Degeneracy in Cross-Validated Skill in Regression-based Forecasts , 1993 .

[16]  Detlef Stammer,et al.  Initializing Decadal Climate Predictions with the GECCO Oceanic Synthesis: Effects on the North Atlantic , 2009 .

[17]  Swadhin K. Behera,et al.  Extended ENSO Predictions Using a Fully Coupled Ocean–Atmosphere Model , 2008 .

[18]  J. Storch,et al.  A Simple Empirical Model for Decadal Climate Prediction , 2011 .

[19]  D. Smith,et al.  Multi‐year predictability of the tropical Atlantic atmosphere driven by the high latitude North Atlantic Ocean , 2011 .

[20]  Renate Hagedorn,et al.  The rationale behind the success of multi-model ensembles in seasonal forecasting-II , 2005 .

[21]  Jochem Marotzke,et al.  Two tales of initializing decadal climate predictions experiments with the ECHAM5/MPI-OM model , 2012 .

[22]  W. Briggs.  Statistical Methods in the Atmospheric Sciences , 2007 .

[23]  R. Knutti,et al.  Spatial-Scale Dependence of Climate Model Performance in the CMIP3 Ensemble , 2011 .

[24]  Alicia R. Karspeck,et al.  A decadal prediction case study: Late twentieth-century North Atlantic Ocean heat content , 2012 .

[25]  H. Storch,et al.  Statistical Analysis in Climate Research , 2000 .

[26]  Vera Pawlowsky-Glahn,et al.  Statistical Modeling , 2007, Encyclopedia of Social Network Analysis and Mining.

[27]  Matthew D. Collins,et al.  A review of predictability studies of Atlantic sector climate on decadal time scales , 2006 .

[28]  David Rind,et al.  How will Earth's surface temperature change in future decades? , 2009 .

[29]  Adam A. Scaife,et al.  Skilful multi-year predictions of Atlantic hurricane frequency , 2010 .

[30]  Ichiro Fukumori,et al.  DISTINGUISHING THE ROLES OF NATURAL AND ANTHROPOGENICALLY FORCED DECADAL CLIMATE VARIABILITY Implications for Prediction , 2011 .

[31]  J. Thepaut,et al.  The ERA‐Interim reanalysis: configuration and performance of the data assimilation system , 2011 .

[32]  P. Linden,et al.  ENSEMBLES: Climate Change and its Impacts - Summary of research and results from the ENSEMBLES project , 2009 .

[33]  T. Stockdale.  Coupled Ocean–Atmosphere Forecasts in the Presence of Climate Drift , 1997 .

[34]  G. Meehl,et al.  Decadal prediction: Can it be skillful? , 2009 .

[35]  L. Kornblueh,et al.  Advancing decadal-scale climate prediction in the North Atlantic sector , 2008, Nature.

[36]  M. Kimoto,et al.  Reevaluation of historical ocean heat content variations with time-varying XBT and MBT depth bias corrections , 2009 .

[37]  Chris Harris,et al.  Improved Atlantic winter blocking in a climate model , 2011 .

[38]  David L. T. Anderson,et al.  Impact of initialization strategies and observations on seasonal forecast skill , 2009 .

[39]  Mark A. Liniger,et al.  Can multi‐model combination really enhance the prediction skill of probabilistic ensemble forecasts? , 2007 .

[40]  K. Wolter,et al.  El Niño/Southern Oscillation behaviour since 1871 as diagnosed in an extended multivariate ENSO index (MEI.ext) , 2011 .

[41]  Karl E. Taylor,et al.  An overview of CMIP5 and the experiment design , 2012 .

[42]  Simon J. Mason.  Seasonal and Longer‐Range Forecasts , 2012 .

[43]  F. Doblas-Reyes,et al.  On the assessment of near-surface global temperature and North Atlantic multi-decadal variability in the ENSEMBLES decadal hindcast , 2012, Climate Dynamics.

[44]  Francisco J. Doblas-Reyes,et al.  Decadal prediction skill in a multi-model ensemble , 2012, Climate Dynamics.

[45]  Adam A. Scaife,et al.  Towards Prediction of Decadal Climate Variability and Change , 2009 .

[46]  B. Efron,et al.  A Leisurely Look at the Bootstrap, the Jackknife, and , 1983 .

[47]  Takashi T. Sakamoto,et al.  Pacific decadal oscillation hindcasts relevant to near-term climate prediction , 2010, Proceedings of the National Academy of Sciences.

[48]  Renate Hagedorn,et al.  The rationale behind the success of multi-model ensembles in seasonal forecasting — I. Basic concept , 2005 .

[49]  N. V. Kobysheva,et al.  Needs Assessment for Climate Information on Decadal Timescales and Longer , 2010 .

[50]  P. Webster,et al.  Evaluation of short‐term climate change prediction in multi‐model CMIP5 decadal hindcasts , 2012 .