Risks of Model Weighting in Multimodel Climate Projections

Multimodel combination is a pragmatic approach to estimating model uncertainties and to making climate projections more reliable. The simplest way of constructing a multimodel is to give one vote to each model (‘‘equal weighting’’), while more sophisticated approaches suggest applying model weights according to some measure of performance (‘‘optimum weighting’’). In this study, a simple conceptual model of climate change projections is introduced and applied to discuss the effects of model weighting in more generic terms. The results confirm that equally weighted multimodels on average outperform the single models, and that projection errors can in principle be further reduced by optimum weighting. However, this not only requires accurate knowledge of the single model skill, but the relative contributions of the joint model error and unpredictable noise also need to be known to avoid biased weights. If weights are applied that do not appropriatelyrepresentthetrueunderlyinguncertainties,weightedmultimodelsperformonaverageworsethan equally weighted ones, which is a scenario that is not unlikely, given that at present there is no consensus on how skill-basedweights can be obtained.Particularly when internal variabilityis large, more information may be lost by inappropriate weighting than could potentially be gained by optimum weighting. These results indicate that for many applications equal weighting may be the safer and more transparent way to combine models. However, also within the presented framework eliminating models from an ensemble can be justified if they are known to lack key mechanisms that are indispensable for meaningful climate projections.

[1]  J. Walsh,et al.  Global Climate Model Performance over Alaska and Greenland , 2008 .

[2]  Mark A. Liniger,et al.  Probabilistic Verification of Monthly Temperature Forecasts , 2008 .

[3]  Upmanu Lall,et al.  Improved Combination of Multiple Atmospheric GCM Ensembles for Seasonal Prediction , 2004 .

[4]  Francisco J. Doblas-Reyes,et al.  Realistic greenhouse gas forcing and seasonal forecasts , 2007 .

[5]  D A Stainforth,et al.  Confidence, uncertainty and decision-support relevance in climate predictions , 2007, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[6]  Mort Webster,et al.  Communicating Climate Change Uncertainty to Policy-Makers and the Public , 2003 .

[7]  M. Webb,et al.  Quantification of modelling uncertainties in a large ensemble of climate change simulations , 2004, Nature.

[8]  H. Künsch,et al.  Bayesian multi-model projection of climate: bias assumptions and interannual variability , 2009 .

[9]  Richard L. Smith,et al.  Quantifying Uncertainty in Projections of Regional Climate Change: A Bayesian Approach to the Analysis of Multimodel Ensembles , 2005 .

[10]  Renate Hagedorn,et al.  The rationale behind the success of multi-model ensembles in seasonal forecasting — I. Basic concept , 2005 .

[11]  D. Nychka,et al.  Spatial Analysis to Quantify Numerical Model Bias and Dependence , 2008 .

[12]  Willem A. Landman,et al.  Climate change 2007 : the physical science basis, S. Solomon, D. Qin, M. Manning, M. Marquis, K. Averyt, M.M.B. Tignor, H. LeRoy Miller, Jr. and Z. Chen (Eds.) : book review , 2010 .

[13]  Upmanu Lall,et al.  Probabilistic multimodel regional temperature change projections , 2006 .

[14]  Jouni Räisänen How reliable are climate models , 2007 .

[15]  Leonard A. Smith,et al.  What might we learn from climate forecasts? , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[16]  Renate Hagedorn,et al.  The rationale behind the success of multi-model ensembles in seasonal forecasting — II. Calibration and combination , 2005 .

[17]  Simon C. Scherrer,et al.  Present‐day interannual variability of surface climate in CMIP3 models and its relation to future warming , 2011 .

[18]  David B. Stephenson,et al.  A Changing Climate for Prediction , 2007, Science.

[19]  P. Whetton,et al.  Assessment of the use of current climate patterns to evaluate regional enhanced greenhouse response patterns of climate models , 2007 .

[20]  Arun Kumar,et al.  Finite Samples and Uncertainty Estimates for Skill Measures for Seasonal Prediction , 2009 .

[21]  Reto Knutti,et al.  The use of the multi-model ensemble in probabilistic climate projections , 2007, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[22]  A. Hall,et al.  September sea-ice cover in the Arctic Ocean projected to vanish by 2100 , 2009 .

[23]  J. Christensen,et al.  On the need for bias correction of regional climate change projections of temperature and precipitation , 2008 .

[24]  René Laprise,et al.  Investigation of regional climate models’ internal variability with a ten-member ensemble of 10-year simulations over a large domain , 2008 .

[25]  Andrew J. Pitman,et al.  Do weak AR4 models bias projections of future climate changes over Australia? , 2009 .

[26]  Edward S. Epstein,et al.  A Scoring System for Probability Forecasts of Ranked Categories , 1969 .

[27]  F. Giorgi,et al.  Probability of regional climate change based on the Reliability Ensemble Averaging (REA) method , 2003 .

[28]  Andrew P. Morse,et al.  DEVELOPMENT OF A EUROPEAN MULTIMODEL ENSEMBLE SYSTEM FOR SEASONAL-TO-INTERANNUAL PREDICTION (DEMETER) , 2004 .

[29]  Mark A. Liniger,et al.  Can multi‐model combination really enhance the prediction skill of probabilistic ensemble forecasts? , 2007 .

[30]  Charles Doutriaux,et al.  Performance metrics for climate models , 2008 .

[31]  Francisco J. Doblas-Reyes,et al.  Forecast assimilation: a unified framework for the combination of multi-model weather and climate predictions , 2005 .

[32]  A. Sterl,et al.  The ERA‐40 re‐analysis , 2005 .

[33]  Francis W. Zwiers,et al.  Improved Seasonal Probability Forecasts , 2003 .

[34]  Jonathan Rougier,et al.  Probabilistic Inference for Future Climate Using an Ensemble of Climate Model Evaluations , 2007 .

[35]  Reto Knutti,et al.  Should we believe model predictions of future climate change? , 2008, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[36]  A. Raftery,et al.  Using Bayesian Model Averaging to Calibrate Forecast Ensembles , 2005 .

[37]  Renate Hagedorn,et al.  The rationale behind the success of multi-model ensembles in seasonal forecasting-II , 2005 .

[38]  Balaji Rajagopalan,et al.  Categorical Climate Forecasts through Regularization and Optimal Combination of Multiple GCM Ensembles , 2002 .

[39]  R. Buizza Potential Forecast Skill of Ensemble Prediction and Spread and Skill Distributions of the ECMWF Ensemble Prediction System , 1997 .

[40]  P. Stott,et al.  External control of 20th century temperature by natural and anthropogenic forcings. , 2000, Science.

[41]  Mark A. Liniger,et al.  Seasonal Ensemble Forecasts: Are Recalibrated Single Models Better than Multimodels? , 2009 .

[42]  Mark A. Liniger,et al.  The discrete brier and ranked probability skill scores , 2007 .

[43]  G. Meehl,et al.  Decadal prediction: Can it be skillful? , 2009 .

[44]  E. Hawkins,et al.  The Potential to Narrow Uncertainty in Regional Climate Predictions , 2009 .

[45]  D J Frame,et al.  Probabilistic climate forecasts and inductive problems , 2007, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[46]  Christopher K. Wikle,et al.  Atmospheric Modeling, Data Assimilation, and Predictability , 2005, Technometrics.

[47]  M. Allen,et al.  Corrigendum: Constraints on future changes in climate and the hydrologic cycle , 2012, Nature.

[48]  F. Giorgi,et al.  An intercomparison of regional climate simulations for Europe: assessing uncertainties in model projections , 2007 .

[49]  Leonard A. Smith,et al.  Evaluating Probabilistic Forecasts Using Information Theory , 2002 .

[50]  K. Popper The Propensity Interpretation of Probability , 1959 .

[51]  R. Toumi,et al.  Climate projections: Past performance no guarantee of future skill? , 2009 .

[52]  Reto Knutti,et al.  Challenges in Combining Projections from Multiple Climate Models , 2010 .

[53]  J. Räisänen,et al.  How reliable are climate models? , 2007 .

[54]  F. Giorgi,et al.  Calculation of average, uncertainty range, and reliability of regional climate changes from AOGCM simulations via the reliability ensemble averaging (REA) method , 2002 .