On the Relationship Between Probabilistic and Deterministic Skills in Dynamical Seasonal Climate Prediction

Despite the gradually increasing emphasis on assessing the skill of dynamical seasonal climate predictions from the probabilistic perspective, there is a lack of in‐depth understanding that an inherent relationship may exist between the probabilistic and deterministic seasonal forecast skills. In this study, we focus on investigating this relationship, through theoretical consideration based on an analytical approach and diagnostic analysis of the historical forecasts produced by multiple dynamical models. The probabilistic forecast skill is gauged in terms of its two different attributes: resolution and reliability, while the deterministic forecast skill is measured in terms of anomaly correlation (AC). Through the theoretical consideration under certain simplified assumptions, a nonlinear, monotonic relationship is analytically derived between the resolution and the AC. Subsequent diagnostic analysis shows that the resolution and AC skills of both the multimodel ensemble and its member single models indeed appear to be approximately monotonically and nonlinearly related, specifically when they are calculated in a zonally aggregated manner by which the impact of finite sample size is reduced. This observed relationship has a specific form that is consistent with what the theory predicts. In short, the theoretical result is well verified by the dynamical model forecasts. Diagnostic analysis also shows that no good relationship exists between the reliability and the AC, signifying the difference of reliability and resolution in nature. A specific application of the proven resolution‐AC coherence is also demonstrated. The proved resolution‐AC relationship can facilitate comparisons among various assessments of seasonal climate prediction skill from the deterministic or probabilistic perspective alone.

[1]  A. Moore,et al.  Measuring the potential predictability of ensemble climate predictions , 2008 .

[2]  Adam A. Scaife,et al.  The Climate-System Historical Forecast Project: Providing Open Access to Seasonal Forecast Ensembles from Centers around the Globe , 2017 .

[3]  Jochen Bröcker,et al.  Estimating reliability and resolution of probability forecasts through decomposition of the empirical score , 2012, Climate Dynamics.

[4]  W. Briggs.  Statistical Methods in the Atmospheric Sciences , 2007 .

[5]  David S. Richardson,et al.  A probability and decision‐model analysis of PROVOST seasonal multi‐model ensemble integrations , 2000 .

[6]  In-Sik Kang,et al.  Dynamic seasonal prediction and predictability of the monsoon , 2006 .

[7]  Adam A. Scaife,et al.  Global Seasonal forecast system version 5 (GloSea5): a high‐resolution seasonal forecast system , 2015 .

[8]  Bin Wang,et al.  How are seasonal prediction skills related to models’ performance on mean state and annual cycle? , 2010 .

[9]  Olivier Talagrand,et al.  The attributes of forecast systems: a general framework for the evaluation and calibration of weather forecasts , 2006 .

[10]  Bin Wang,et al.  Season-Dependent Forecast Skill of the Leading Forced Atmospheric Circulation Pattern over the North Pacific and North American Region* , 2012 .

[11]  A. Barnston.  Correspondence among the correlation, RMSE, and Heidke forecast verification measures; refinement of the Heidke score , 1992 .

[12]  Jeffrey L. Anderson,et al.  Reproducible Forced Modes in AGCM Ensemble Integrations and Potential Predictability of Atmospheric Seasonal Variations in the Extratropics , 1998 .

[13]  Tim N. Palmer,et al.  The economic value of ensemble forecasts as a tool for risk assessment: From days to decades , 2002 .

[14]  B. Kirtman,et al.  Predictable skill and its association to sea‐surface temperature variations in an ensemble climate simulation , 2005 .

[15]  Bin Wang,et al.  Fundamental challenge in simulation and prediction of summer monsoon rainfall , 2005 .

[16]  Wei Zhang,et al.  Improved ENSO Forecasting using Bayesian Updating and the North American Multi Model Ensemble (NMME) , 2017 .

[17]  M. Rodwell,et al.  Oceanic forcing of the wintertime North Atlantic Oscillation and European climate , 1999, Nature.

[18]  Youmin Tang,et al.  An analysis of multi‐model ensembles for seasonal climate predictions , 2013 .

[19]  Timothy DelSole,et al.  Predictability and Information Theory. Part II: Imperfect Forecasts , 2005 .

[20]  Mark A. Liniger,et al.  Seasonal Ensemble Forecasts: Are Recalibrated Single Models Better than Multimodels? , 2009 .

[21]  Qiaoping Li,et al.  Performance of the seasonal forecasting of the Asian summer monsoon by BCC_CSM1.1(m) , 2015, Advances in Atmospheric Sciences.

[22]  Huug van den Dool,et al.  An analysis of multimodel ensemble predictions for seasonal climate anomalies , 2002 .

[23]  J. Shukla,et al.  The Role of Air–Sea Coupling in Seasonal Prediction of Asia–Pacific Summer Monsoon Rainfall , 2013 .

[24]  Roberto Buizza,et al.  The new ECMWF seasonal forecast system (system 4) , 2011 .

[25]  F. Molteni,et al.  Atmospheric initial conditions and the predictability of the Arctic Oscillation , 2015 .

[26]  C. M. Kishtawal,et al.  Multimodel Ensemble Forecasts for Weather and Seasonal Climate , 2000 .

[27]  Francis W. Zwiers,et al.  Skill assessment of seasonal hindcasts from the Canadian historical forecast project , 2009 .

[28]  David S. Richardson,et al.  Predictability of Weather and Climate: Predictability and economic value , 2006 .

[29]  Dake Chen,et al.  Ensemble Construction and Verification of the Probabilistic ENSO Prediction in the LDEO5 Model , 2010 .

[30]  David P. Rowell,et al.  Assessing Potential Seasonal Predictability with an Ensemble of Multidecadal GCM Simulations , 1998 .

[31]  Renate Hagedorn,et al.  The rationale behind the success of multi-model ensembles in seasonal forecasting — I. Basic concept , 2005 .

[32]  T. Palmer,et al.  ENSEMBLES: A new multi‐model ensemble for seasonal‐to‐annual predictions—Skill and progress beyond DEMETER in forecasting tropical Pacific SSTs , 2009 .

[33]  Jong-Seong Kug,et al.  Seasonal climate predictability with Tier-one and Tier-two prediction systems , 2008 .

[34]  Adam A. Scaife,et al.  The Climate‐system Historical Forecast Project: do stratosphere‐resolving models make better seasonal climate predictions in boreal winter? , 2015 .

[35]  Thomas M. Hamill,et al.  Measuring forecast skill: is it real skill or is it the varying climatology? , 2006 .

[36]  Xiu‐Qun Yang,et al.  Probabilistic versus deterministic skill in predicting the western North Pacific‐East Asian summer monsoon variability with multimodel ensembles , 2015 .

[37]  Bin Wang,et al.  Predictability of summer northwest Pacific climate in 11 coupled model hindcasts: Local and remote forcing , 2010 .

[38]  Arun Kumar,et al.  NCEP dynamical seasonal forecast system 2000 , 2002 .

[39]  Swadhin K. Behera,et al.  Extended ENSO Predictions Using a Fully Coupled Ocean–Atmosphere Model , 2008 .

[40]  Timothy DelSole,et al.  Model Fidelity versus Skill in Seasonal Forecasting , 2010 .

[41]  J. G. Charney,et al.  Monsoon dynamics: Predictability of monsoons , 1981 .

[42]  North American Region , 2014 .

[43]  Francis W. Zwiers,et al.  Improved Seasonal Probability Forecasts , 2003 .

[44]  H. V. D. Dool,et al.  Probabilistic Seasonal Forecasts in the North American Multimodel Ensemble: A Baseline Skill Assessment , 2016 .

[45]  A. Barnston,et al.  The North American multimodel ensemble: Phase-1 seasonal-to-interannual prediction; phase-2 toward developing intraseasonal prediction , 2014 .

[46]  F. Doblas-Reyes,et al.  Multi-model seasonal hindcasts over the Euro-Atlantic: skill scores and dynamic features , 2000 .

[47]  Xiu‐Qun Yang,et al.  Information-based potential predictability of the Asian summer monsoon in a coupled model , 2012 .

[48]  Michael K. Tippett,et al.  Assessing probabilistic predictions of ENSO phase and intensity from the North American Multimodel Ensemble , 2017, Climate Dynamics.

[49]  Arun Kumar,et al.  A Comprehensive Assessment of CFS Seasonal Forecasts over the Tropics , 2012 .

[50]  Daniel S. Wilks,et al.  Smoothing forecast ensembles with fitted probability distributions , 2002 .

[51]  A. Cofiño,et al.  Validation of 40 year multimodel seasonal precipitation forecasts: The role of ENSO on the global skill , 2014 .

[52]  On the comparison between seasonal predictive skill of global circulation models: Coupled versus uncoupled , 2015 .

[53]  Adam A. Scaife,et al.  Tropical rainfall, Rossby waves and regional winter climate predictions , 2017 .

[54]  Arun Kumar,et al.  Understanding Prediction Skill of Seasonal Mean Precipitation over the Tropics , 2013 .

[55]  Renate Hagedorn,et al.  The rationale behind the success of multi-model ensembles in seasonal forecasting — II. Calibration and combination , 2005 .

[56]  Bin Wang,et al.  Deficiencies and possibilities for long-lead coupled climate prediction of the Western North Pacific-East Asian summer monsoon , 2011 .

[57]  R. T. Barnes,et al.  A performance comparison of coupled and uncoupled versions of the Met Office seasonal prediction general circulation model , 2005 .

[58]  Bin Wang,et al.  Advance and prospectus of seasonal prediction: assessment of the APCC/CliPAS 14-model ensemble retrospective seasonal prediction (1980–2004) , 2009 .

[59]  Antje Weisheimer,et al.  Evaluation of Probabilistic Quality and Value of the ENSEMBLES Multimodel Seasonal Forecasts: Comparison with DEMETER , 2011 .

[60]  Zoltan Toth,et al.  Why Do Forecasts for “Near Normal” Often Fail? , 1991 .

[61]  R. Lu,et al.  Predictability of the western North Pacific summer climate associated with different ENSO phases by ENSEMBLES multi-model seasonal forecasts , 2014, Climate Dynamics.

[62]  Kenneth R. Sperber,et al.  Interannual Tropical Rainfall Variability in General Circulation Model Simulations Associated with the Atmospheric Model Intercomparison Project , 1996 .

[63]  Bin Wang,et al.  How predictable is the northern hemisphere summer upper-tropospheric circulation? , 2011 .

[64]  Phillip A. Arkin,et al.  Analyses of Global Monthly Precipitation Using Gauge Observations, Satellite Estimates, and Numerical Model Predictions , 1996 .

[65]  Jochen Bröcker,et al.  Resolution and discrimination–two sides of the same coin , 2015 .

[66]  David S. Richardson,et al.  Predictability and economic value , 2003 .

[67]  S. Saha,et al.  The NCEP Climate Forecast System , 2006 .

[68]  D. Richardson.  Skill and relative economic value of the ECMWF ensemble prediction system , 2000 .

[69]  Youmin Tang,et al.  The Canadian Seasonal to Interannual Prediction System. Part I: Models and Initialization , 2013 .

[70]  In-Sik Kang,et al.  Theoretical examination of a multi‐model composite for seasonal prediction , 2005 .

[71]  M. Iredell,et al.  The NCEP Climate Forecast System Version 2 , 2014 .

[72]  T. N. Krishnamurti,et al.  Improved Weather and Seasonal Climate Forecasts from Multimodel Superensemble. , 1999, Science.

[73]  Venkatramani Balaji,et al.  Improved Seasonal Prediction of Temperature and Precipitation over Land in a High-Resolution GFDL Climate Model , 2015 .

[74]  M. Balmaseda,et al.  Global seasonal rainfall forecasts using a coupled ocean–atmosphere model , 1998, Nature.

[75]  Timothy DelSole,et al.  Predictability and Information Theory. Part I: Measures of Predictability , 2004 .

[76]  F. Atger,et al.  Estimation of the reliability of ensemble‐based probabilistic forecasts , 2004 .

[77]  Andrew P. Morse,et al.  DEVELOPMENT OF A EUROPEAN MULTIMODEL ENSEMBLE SYSTEM FOR SEASONAL-TO-INTERANNUAL PREDICTION (DEMETER) , 2004 .

[78]  Mark A. Liniger,et al.  Can multi‐model combination really enhance the prediction skill of probabilistic ensemble forecasts? , 2007 .

[79]  Actual and potential skill of seasonal predictions using the CNRM contribution to DEMETER: coupled versus uncoupled model , 2005 .

[80]  Prashant D. Sardeshmukh,et al.  Changes of Probability Associated with El Niño , 2000 .

[81]  A Postprocessing Method for Seasonal Forecasts Using Temporally and Spatially Smoothed Statistics , 2017 .

[82]  Bin Wang,et al.  Seasonal prediction and predictability of the Asian winter temperature variability , 2013, Climate Dynamics.

[83]  Dong Eun Lee,et al.  Seasonal Rainfall Prediction Skill over South Africa: One- versus Two-Tiered Forecasting Systems , 2012 .

[84]  A. Barnston,et al.  Changes in the Spread of the Variability of the Seasonal Mean Atmospheric States Associated with ENSO , 2000 .

[85]  F. Molteni,et al.  A rainfall calibration methodology for impacts modelling based on spatial mapping , 2012 .

[86]  J. Whitaker,et al.  Diagnosing Sources of U.S. Seasonal Forecast Skill , 2006 .

[87]  Bin Wang,et al.  Assessment of the long-lead probabilistic prediction for the Asian summer monsoon precipitation (1983-2011) based on the APCC multimodel system and a statistical model , 2012 .

[88]  Young-Mi Min,et al.  Skill of real-time operational forecasts with the APCC multi-model ensemble prediction system during the period 2008–2015 , 2017, Climate Dynamics.

[89]  J. Shukla,et al.  Predictability in the midst of chaos: A scientific basis for climate forecasting , 1998, Science.

[90]  R. Reynolds,et al.  The NCEP/NCAR 40-Year Reanalysis Project , 1996, Renewable Energy.