Model Fidelity versus Skill in Seasonal Forecasting

Abstract The relation between skill and fidelity of seasonal mean hindcasts of surface temperature by seven coupled atmosphere–ocean models is investigated. By definition, fidelity measures the agreement between model and observational climatological distributions, and skill measures the agreement between hindcasts and their corresponding verifications. While a relation between skill and fidelity seems intuitively plausible, it has not been checked systematically, nor is it mandated mathematically. New measures of skill and fidelity based on information theory are proposed. Specifically, fidelity is measured by the area average relative entropy between the climatological distributions of the model and observations, and skill is measured by the area averaged mutual information between forecast and verification. The fidelity measure is found to be dominated by the term measuring mean bias; that is, the discrepancy in climatological means is much larger than the discrepancy in climatological variances. Moreo...

[1]  Thomas M. Smith,et al.  Specification and Prediction of Global Surface Temperature and Precipitation from Global SST Using CCA , 1996 .

[2]  F. Giorgi,et al.  Calculation of average, uncertainty range, and reliability of regional climate changes from AOGCM simulations via the reliability ensemble averaging (REA) method , 2002 .

[3]  Timothy DelSole,et al.  Empirical Correction of the NCEP Global Forecast System , 2008 .

[4]  K. Swanson,et al.  Storm Track Dynamics , 2002, The Global Circulation of the Atmosphere.

[5]  Mei Zhao,et al.  Empirical Correction of a Coupled Land-Atmosphere Model , 2008 .

[6]  Andrew P. Morse,et al.  DEVELOPMENT OF A EUROPEAN MULTIMODEL ENSEMBLE SYSTEM FOR SEASONAL-TO-INTERANNUAL PREDICTION (DEMETER) , 2004 .

[7]  K. Hasselmann.  On the signal-to-noise problem in atmospheric response studies , 1979 .

[8]  J. Shukla,et al.  Climate model fidelity and projections of climate change , 2006 .

[9]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[10]  Cma Training,et al.  Predictability of Weather and Climate , 2011 .

[11]  H. Joe.  Relative Entropy Measures of Multivariate Dependence , 1989 .

[12]  Timothy DelSole,et al.  Predictability and Information Theory. Part II: Imperfect Forecasts , 2005 .

[13]  Richard L. Smith,et al.  Quantifying Uncertainty in Projections of Regional Climate Change: A Bayesian Approach to the Analysis of Multimodel Ensembles , 2005 .

[14]  M. Kanamitsu,et al.  NCEP–DOE AMIP-II Reanalysis (R-2) , 2002 .

[15]  Timothy DelSole,et al.  Predictability and Information Theory. Part I: Measures of Predictability , 2004 .

[16]  Richard L. Smith,et al.  Regional probabilities of precipitation change: A Bayesian analysis of multimodel simulations , 2004 .

[17]  F. Zwiers,et al.  Climate Predictions with Multimodel Ensembles , 2002 .

[18]  Renate Hagedorn,et al.  The rationale behind the success of multi-model ensembles in seasonal forecasting — I. Basic concept , 2005 .

[19]  Renate Hagedorn,et al.  The rationale behind the success of multi-model ensembles in seasonal forecasting-II , 2005 .

[20]  Andrew T. Wittenberg,et al.  How Predictable is El Niño , 2003 .

[21]  Benjamin Kirtman,et al.  Decadal Variability in ENSO Predictability and Prediction , 1998 .

[22]  J. Shukla,et al.  Specification of Wintertime North American Surface Temperature , 2006 .

[23]  T. DelSole,et al.  Stochastic Models of Quasigeostrophic Turbulence , 2004 .

[24]  Michael J. Rycroft,et al.  Storms in Space , 2004 .