Multi-model calibration and combination of tropical seasonal sea surface temperature forecasts

Different combination methods based on multiple linear regression are explored to identify the conditions that lead to an improvement of seasonal forecast quality when individual operational dynamical systems and a statistical–empirical system are combined. A calibration of the post-processed output is included. The combination methods have been used to merge the ECMWF System 4, the NCEP CFSv2, the Météo-France System 3, and a simple statistical model based on SST lagged regression. The forecast quality was assessed from a deterministic and probabilistic point of view. SSTs averaged over three different tropical regions have been considered: the Niño3.4, the Subtropical Northern Atlantic and Western Tropical Indian SST indices. The forecast quality of these combinations is compared to the forecast quality of a simple multi-model (SMM) where all single models are equally weighted. The results show a large range of behaviours depending on the start date, target month and the index considered. Outperforming the SMM predictions is a difficult task for linear combination methods with the samples currently available in an operational context. The difficulty in the robust estimation of the weights due to the small samples available is one of the reasons that limit the potential benefit of the combination methods that assign unequal weights. However, these combination methods showed the capability to improve the forecast reliability and accuracy in a large proportion of cases. For example, the Forecast Assimilation method proved to be competitive against the SMM while the other combination methods outperformed the SMM when only a small number of forecast systems have skill. Therefore, the weighting does not outperform the SMM when the SMM is very skilful, but it reduces the risk of low skill situations that are found when several single forecast systems have a low skill.

[1]  Li Zhang,et al.  An Analysis of the Nonstationarity in the Bias of Sea Surface Temperature Forecasts for the NCEP Climate Forecast System (CFS) Version 2 , 2012 .

[2]  Simon J. Mason,et al.  Understanding forecast verification statistics , 2008 .

[3]  Simon J. Mason,et al.  Comparison of Some Statistical Methods of Probabilistic Forecasting of ENSO. , 2002 .

[4]  A. Barnston,et al.  A Degeneracy in Cross-Validated Skill in Regression-based Forecasts , 1993 .

[5]  A. H. Murphy,et al.  Probability Forecasting in Meteorology , 1984 .

[6]  Lifeng Luo,et al.  A first look at Climate Forecast System version 2 (CFSv2) for hydrological seasonal prediction , 2011 .

[7]  J. Shukla,et al.  Predictability in the midst of chaos: A scientific basis for climate forecasting , 1998, Science.

[8]  David L. T. Anderson,et al.  Decadal and Seasonal Dependence of ENSO Prediction Skill , 1995 .

[9]  Adrian E. Raftery,et al.  Weather Forecasting with Ensemble Methods , 2005, Science.

[10]  S. Saha,et al.  The NCEP Climate Forecast System , 2006 .

[11]  L. Batté,et al.  Seasonal predictions of precipitation over Africa using coupled ocean-atmosphere general circulation models: skill of the ENSEMBLES project multimodel ensemble forecasts , 2011 .

[12]  Andrew P. Morse,et al.  DEVELOPMENT OF A EUROPEAN MULTIMODEL ENSEMBLE SYSTEM FOR SEASONAL-TO-INTERANNUAL PREDICTION (DEMETER) , 2004 .

[13]  A Comprehensive Assessment of CFS Seasonal Forecasts over the Tropics , 2012 .

[14]  In-Sik Kang,et al.  Optimal Multi-model Ensemble Method in Seasonal Climate Prediction , 2008 .

[15]  Reto Knutti,et al.  The end of model democracy? , 2010 .

[16]  Michael K. Tippett,et al.  Skill of Multimodel ENSO Probability Forecasts , 2008 .

[17]  Bin Wang,et al.  Advance and prospectus of seasonal prediction: assessment of the APCC/CliPAS 14-model ensemble retrospective seasonal prediction (1980–2004) , 2009 .

[18]  David L. T. Anderson,et al.  Seasonal Climate: Forecasting and Managing Risk , 2008 .

[19]  Kristian Mogensen,et al.  ECMWF seasonal forecast system 3 and its prediction of sea surface temperature , 2011 .

[20]  Francisco J. Doblas-Reyes,et al.  Seasonal climate predictability and forecasting: status and prospects , 2013 .

[21]  M. Iredell,et al.  The NCEP Climate Forecast System Version 2 , 2014 .

[22]  C. Ropelewski,et al.  Current approaches to seasonal to interannual climate predictions , 2001 .

[23]  C. Ropelewski,et al.  Global and Regional Scale Precipitation Patterns Associated with the El Niño/Southern Oscillation , 1987 .

[24]  Renate Hagedorn,et al.  The rationale behind the success of multi-model ensembles in seasonal forecasting-II , 2005 .

[25]  Jong-Seong Kug,et al.  Global Sea Surface Temperature Prediction Using a Multimodel Ensemble , 2007 .

[26]  T. Palmer.  Predicting uncertainty in forecasts of weather and climate , 2000 .

[27]  Elizabeth C. Kent,et al.  Global analyses of sea surface temperature, sea ice, and night marine air temperature since the late nineteenth century , 2003 .

[28]  Carlos A. Coelho,et al.  Forecast calibration and combination: A simple Bayesian approach for ENSO , 2004 .

[29]  Vera Pawlowsky-Glahn,et al.  Statistical Modeling , 2007, Encyclopedia of Social Network Analysis and Mining.

[30]  Roberto Buizza,et al.  The new ECMWF seasonal forecast system (system 4) , 2011 .

[31]  Peter J. Webster,et al.  Climate Science and the Uncertainty Monster , 2011 .

[32]  T. Palmer,et al.  Addressing model uncertainty in seasonal and annual dynamical ensemble forecasts , 2009 .

[33]  Peter J. Webster,et al.  Seasonal prediction skill of ECMWF System 4 and NCEP CFSv2 retrospective forecast for the Northern Hemisphere Winter , 2012, Climate Dynamics.

[34]  M. Tippett,et al.  Is unequal weighting significantly better than equal weighting for multi‐model forecasting? , 2013 .

[35]  Renate Hagedorn,et al.  The rationale behind the success of multi-model ensembles in seasonal forecasting — I. Basic concept , 2005 .

[36]  Tim Palmer,et al.  Uncertainty in weather and climate prediction , 2011, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[37]  I. Jolliffe,et al.  Two Extra Components in the Brier Score Decomposition , 2008 .

[38]  Francisco J. Doblas-Reyes,et al.  Forecast assimilation: a unified framework for the combination of multi-model weather and climate predictions , 2005 .

[39]  David B. Stephenson,et al.  How Do We Know Whether Seasonal Climate Forecasts are Any Good , 2008 .

[40]  B. Goswami,et al.  A dipole mode in the tropical Indian Ocean , 1999, Nature.