Using climate models to estimate the quality of global observational data sets

Models and data: A two-way street Data are used to drive models of climate and other complex systems, but is the relationship between data and models a one-way process? Massonnet et al. used climate models to assess the quality of the observations that such models use. Starting with a simple model and progressing to more complex ones, the authors show that models are better when they are assessed against the most recent, most advanced, and most independent observational references. These findings should help to evaluate the quality of observational data sets and provide guidance for more objective data set selection. Science, this issue p. 452 Climate models can be used to assess the quality of the observational data sets they use. Observational estimates of the climate system are essential to monitoring and understanding ongoing climate change and to assessing the quality of climate models used to produce near- and long-term climate information. This study poses the dual and unconventional question: Can climate models be used to assess the quality of observational references? We show that this question not only rests on solid theoretical grounds but also offers insightful applications in practice. By comparing four observational products of sea surface temperature with a large multimodel climate forecast ensemble, we find compelling evidence that models systematically score better against the most recent, advanced, but also most independent product. These results call for generalized procedures of model-observation comparison and provide guidance for a more objective observational data set selection.

[1]  Elizabeth C. Kent,et al.  Global analyses of sea surface temperature, sea ice, and night marine air temperature since the late nineteenth century , 2003 .

[2]  Masson-Delmotte,et al.  The Physical Science Basis , 2007 .

[3]  K. Taylor.  Summarizing multiple aspects of model performance in a single diagram , 2001 .

[4]  S. Kern,et al.  Inter-comparison and evaluation of sea ice algorithms: towards further identification of challenges and optimal approach using passive microwave observations , 2015 .

[5]  Charles Doutriaux,et al.  Performance metrics for climate models , 2008 .

[6]  M. Kimoto,et al.  Initialized near-term regional climate change prediction , 2013, Nature Communications.

[7]  Saleh Abdalla,et al.  Error Estimation of Buoy, Satellite, and Model Wave Height Data , 2007 .

[8]  Christopher K. Wikle,et al.  Atmospheric Modeling, Data Assimilation, and Predictability , 2005, Technometrics.

[9]  F. Fetterer,et al.  Sea ice index monitors polar ice extent , 2004 .

[10]  Mark A. Liniger,et al.  Can multi‐model combination really enhance the prediction skill of probabilistic ensemble forecasts? , 2007 .

[11]  Gu Xingfa,et al.  Overview of China Earth Observation Satellite Programs [Space Agencies] , 2015, IEEE Geoscience and Remote Sensing Magazine.

[12]  P. Cox,et al.  Quantifying future climate change , 2012 .

[13]  Veronika Eyring,et al.  Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organization , 2015 .

[14]  F. Doblas-Reyes,et al.  Ensemble of sea ice initial conditions for interannual climate predictions , 2014, Climate Dynamics.

[15]  W. Briggs.  Statistical Methods in the Atmospheric Sciences , 2007 .

[16]  J. H. Steiger.  Statistically based tests for the number of common factors , 1980 .

[17]  E. Guilyardi,et al.  ENSO representation in climate models: from CMIP3 to CMIP5 , 2013, Climate Dynamics.

[18]  Francisco J. Doblas-Reyes,et al.  Attribution of extreme weather and climate events overestimated by unreliable climate simulations , 2016 .

[19]  G. Hegerl.  Use of models and observations in event attribution , 2015 .

[20]  T. Fichefet,et al.  On the influence of model physics on simulations of Arctic and Antarctic sea ice , 2011 .

[21]  A. Barnston,et al.  The North American multimodel ensemble: Phase-1 seasonal-to-interannual prediction; phase-2 toward developing intraseasonal prediction , 2014 .

[22]  J. Thepaut,et al.  The ERA‐Interim reanalysis: configuration and performance of the data assimilation system , 2011 .

[23]  Thomas M. Smith,et al.  Extended Reconstructed Sea Surface Temperature Version 4 (ERSST.v4). Part I: Upgrades and Intercomparisons , 2014 .

[24]  Stefan Siegert,et al.  A Bayesian framework for verification and recalibration of ensemble forecasts: How uncertain is NAO predictability? , 2015, 1504.01933.

[25]  Adam A. Scaife,et al.  Skillful long‐range prediction of European and North American winters , 2014 .

[26]  T. Reichler,et al.  How Well Do Coupled Models Simulate Today's Climate? , 2008 .

[27]  Claire E. Bulgin,et al.  Sea surface temperature datasets for climate applications from Phase 1 of the European Space Agency Climate Change Initiative (SST CCI) , 2014 .

[28]  Reto Knutti,et al.  On the interpretation of constrained climate model ensembles , 2012 .

[29]  Francisco J. Doblas-Reyes,et al.  Unreliable climate simulations overestimate attributable risk of extreme weather and climate events , 2016 .