Review on ESMValTool v2.0 – Extended set of large-scale diagnostics for quasi-operational and comprehensive evaluation of Earth system models in CMIP

Abstract. The Earth System Model Evaluation Tool (ESMValTool) is a community diagnostics and performance metrics tool designed to improve comprehensive and routine evaluation of Earth System Models (ESMs) participating in the Coupled Model Intercomparison Project (CMIP). It has undergone rapid development since the first release in 2016 and is now a well-tested tool that provides end-to-end provenance tracking to ensure reproducibility. It consists of an easy-to-install, well documented Python package providing the core functionalities (ESMValCore) that performs common pre-processing operations and a diagnostic part that includes tailored diagnostics and performance metrics for specific scientific applications. Here we describe large-scale diagnostics of the second major release of the tool that supports the evaluation of ESMs participating in CMIP Phase 6 (CMIP6). ESMValTool v2.0 includes a large collection of diagnostics and performance metrics for atmospheric, oceanic, and terrestrial variables for the mean state, trends, and variability. ESMValTool v2.0 also successfully reproduces figures from the evaluation and projections chapters of the Intergovernmental Panel on Climate Change (IPCC) Fifth Assessment Report (AR5) and incorporates updates from targeted analysis packages, such as the NCAR Climate Variability Diagnostics Package for the evaluation of modes of variability the Thermodynamic Diagnostic Tool (TheDiaTo) to evaluate the energetics of the climate system, as well as parts of AutoAssess that contains a mix of top-down performance metrics. The tool has been fully integrated into the Earth System Grid Federation (ESGF) infrastructure at the Deutsches Klima Rechenzentrum (DKRZ) to provide evaluation results from CMIP6 model simulations shortly after the output is published to the CMIP archive. A result browser has been implemented that enables advanced monitoring of the evaluation results by a broad user community at much faster timescales than what was possible in CMIP5.

Nuno Carvalhais | Klaus Zimmermann | Lisa Bock | François Massonnet | Bouwe Andela | Björn Brötz | Paul Earnshaw | Birgit Hassler | Axel Lauer | Lee de Mora | Valeriu Predoi | Manuel Schlund | Javier Vegas-Regidor | Ranjini Swaminathan | Nikolay V. Koldunov | Núria Pérez-Zanón | Alasdair Hunter | Carsten Ehbrecht | Nicola Cortesi | Stephan Kindermann | Benjamin Müller | Irene Cionni | Paolo Davini | Mattia Righi | Veronika Eyring | Enrico Arnone | Omar Bellprat | Bas Crezee | Edouard Davin | Kevin Debeire | Clara Deser | David Docquier | Bettina K. Gier | Nube Gonzalez-Reviriego | Paul J. Goodman | Stefan Hagemann | Steven C. Hardiman | Christopher Kadow | Sujan Koirala | Quentin Lejeune | Valerio Lembo | Tomas Lovato | Valerio Lucarini | Amarjiit Pandde | Adam S. Phillips | Joellen Russell | Alistair Sellar | Federico Serva | Tobias Stacke | Verónica Torralba | Jost von Hardenberg | Katja Weigel | Louis-Phillippe Caron | C. Deser | A. Phillips | S. Hagemann | R. Swaminathan | V. Predoi | J. von Hardenberg | T. Stacke | P. Earnshaw | N. Cortesi | V. Torralba | N. González-Reviriego | E. Davin | N. Carvalhais | V. Eyring | I. Cionni | M. Righi | A. Lauer | S. Hardiman | A. Sellar | Sujan Koirala | K. Weigel | V. Lucarini | C. Kadow | C. Ehbrecht | S. Kindermann | T. Lovato | L. Caron | J. Russell | F. Massonnet | P. Davini | O. Bellprat | M. Schlund | Kevin Debeire | L. Bock | B. Andela | E. Arnone | Alasdair Hunter | N. Pérez-Zañón | Javier Vegas-Regidor | B. Hassler | V. Lembo | L. de Mora | D. Docquier | P. Goodman | F. Serva | Q. Lejeune | B. Crezee | Klaus Zimmermann | Benjamin Müller | Björn Brötz | Amarjiit Pandde | Nikolay Koldunov | Manuel Schlund

[1]  J. Wallace,et al.  Teleconnections in the Geopotential Height Field during the Northern Hemisphere Winter , 1981 .

[2]  Karen A. McKinnon,et al.  The Northern Hemisphere Extratropical Atmospheric Circulation Response to ENSO: How Well Do We Know It and How Do We Evaluate Models Accordingly? , 2017 .

[3]  Fabio D'Andrea,et al.  Northern Hemisphere Atmospheric Blocking Representation in Global Climate Models: Twenty Years of Improvements? , 2016 .

[4]  Alessandro Cescatti,et al.  Biophysical climate impacts of recent changes in global forest cover , 2016, Science.

[5]  Marilyn N. Raphael,et al.  Atmospheric influences on the anomalous 2016 Antarctic sea ice decay , 2017 .

[6]  Paolo Davini,et al.  Bidimensional Diagnostics, Variability, and Trends of Northern Hemisphere Blocking , 2012 .

[7]  Laura Ferranti,et al.  Flow‐dependent verification of the ECMWF ensemble over the Euro‐Atlantic sector , 2015 .

[8]  P. Cox,et al.  Evaluating the Land and Ocean Components of the Global Carbon Cycle in the CMIP5 Earth System Models , 2013 .

[9]  Klaus Zimmermann,et al.  ESMValTool v2.0 – Technical overview , 2019 .

[10]  Martin Wild,et al.  Energy budgets and transports: global evolution and spatial patterns during the twentieth century as estimated in two AMIP-like experiments , 2017, Climate Dynamics.

[11]  Valerio Lucarini,et al.  Entropy production and coarse graining of the climate fields in a general circulation model , 2013, Climate Dynamics.

[12]  Simon Read,et al.  ESMValTool (v1.0) – a community diagnostic and performance metrics tool for routine evaluation of Earth system models in CMIP , 2015 .

[13]  David M. Romps,et al.  The Dry-Entropy Budget of a Moist Atmosphere , 2008 .

[14]  D. Barriopedro,et al.  Application of blocking diagnosis methods to General Circulation Models. Part I: a novel detection scheme , 2010 .

[15]  Frank Lunkeit,et al.  Diagnosing the entropy budget of a climate model , 2008 .

[16]  E. Davin,et al.  Biogeophysical impacts of forestation in Europe: First results from the LUCAS Regional Climate Model intercomparison , 2019 .

[17]  Daniel F. Rex,et al.  Blocking Action in the Middle Troposphere and its Effect upon Regional Climate I. An Aerological Study of Blocking Action. , 1950 .

[18]  Elizabeth A. Barnes,et al.  Modeled and Observed Multidecadal Variability in the North Atlantic Jet Stream and Its Connection to Sea Surface Temperatures , 2018 .

[19]  T. Reichler,et al.  How Well Do Coupled Models Simulate Today's Climate? , 2008 .

[20]  L. Bengtsson,et al.  Can an ensemble climate simulation be used to separate climate change signals from internal unforced variability? , 2018, Climate Dynamics.

[21]  P. Dirmeyer,et al.  University of Nebraska-Lincoln DigitalCommons @ University of Nebraska-Lincoln Papers in Natural Resources Natural Resources , School of 2014 Land cover changes and their biogeophysical effects on climate , 2016 .

[22]  J. Wallace,et al.  Annular Modes in the Extratropical Circulation. Part I: Month-to-Month Variability* , 2000 .

[23]  Alessandro Cescatti,et al.  A dataset mapping the potential biophysical effects of vegetation cover change , 2018, Scientific Data.

[24]  M. Heimann,et al.  Terrestrial ecosystem carbon dynamics and climate feedbacks , 2008, Nature.

[25]  Valerio Lucarini,et al.  TheDiaTo (v1.0) – a new diagnostic tool for water, energy and entropy budgets in climate models , 2019, Geoscientific Model Development.

[26]  Martin Jung,et al.  The FLUXCOM ensemble of global land-atmosphere energy fluxes , 2018, Scientific Data.

[27]  Uwe Ulbrich,et al.  The global energy cycle of stationary and transient atmospheric waves: Results from ECMWF analyses , 1991 .

[28]  P. Cox,et al.  Emergent constraints on climate‐carbon cycle feedbacks in the CMIP5 Earth system models , 2014 .

[29]  Sergey Danilov,et al.  An assessment of the Arctic Ocean in a suite of interannual CORE-II simulations. Part II: Liquid freshwater , 2016 .

[30]  M. Iredell,et al.  The NCEP Climate Forecast System Version 2 , 2014 .

[31]  Stefan Rahmstorf,et al.  A decade of weather extremes , 2012 .

[32]  P. Jones,et al.  Quantifying uncertainties in global and regional temperature change using an ensemble of observational estimates: The HadCRUT4 data set , 2012 .

[33]  Valerio Lucarini,et al.  ENERGETICS OF CLIMATE MODELS: NET ENERGY BALANCE AND MERIDIONAL ENTHALPY TRANSPORT , 2009, 0911.5689.

[34]  James W. Hurrell,et al.  North Atlantic climate variability: The role of the North Atlantic Oscillation , 2009 .

[35]  Gregory Duveiller,et al.  The mark of vegetation change on Earth’s surface energy balance , 2018, Nature Communications.

[36]  Brian J. Hoskins,et al.  Winter and Summer Northern Hemisphere Blocking in CMIP5 Models , 2013 .

[37]  Franco Molteni,et al.  On the operational predictability of blocking , 1990 .

[38]  J. Janowiak,et al.  The Version 2 Global Precipitation Climatology Project (GPCP) Monthly Precipitation Analysis (1979-Present) , 2003 .

[39]  Axel Lauer,et al.  Earth System Model Evaluation Tool (ESMValTool) v2.0 – diagnostics for emergent constraints and future projections from Earth system models in CMIP , 2020, Geoscientific Model Development.

[40]  C. Deser,et al.  Communication of the role of natural variability in future North American climate , 2012 .

[41]  Eric Guilyardi,et al.  Towards improved and more routine Earth system model evaluation in CMIP , 2016 .

[42]  D. Klocke,et al.  Tuning the climate of a global model , 2012 .

[43]  S. Gorshkov,et al.  World ocean atlas , 1976 .

[44]  R. Schnur,et al.  Climate-carbon cycle feedback analysis: Results from the C , 2006 .

[45]  Ralph D. Lorenz,et al.  Non-equilibrium thermodynamics and the production of entropy : life, earth, and beyond , 2005 .

[46]  Veronika Eyring,et al.  A Strategy for Process-Oriented Validation of Coupled Chemistry- Climate Models , 2005 .

[47]  W. Collins,et al.  Evaluation of climate models , 2013 .

[48]  Michel Rixen,et al.  The CMIP6 Data Request (version 01.00.31) , 2019 .

[49]  Benjamin Müller,et al.  Benchmarking CMIP5 models with a subset of ESA CCI Phase 2 data using the ESMValTool , 2017 .

[50]  Pierre Friedlingstein,et al.  Controls on terrestrial carbon feedbacks by productivity versus turnover in the CMIP5 Earth System Models , 2015 .

[51]  Maurizio Santoro,et al.  Global covariation of carbon turnover times with climate in terrestrial ecosystems , 2014, Nature.

[52]  C. Deser,et al.  Projecting North American Climate over the Next 50 Years: Uncertainty due to Internal Variability* , 2014 .

[53]  Veronika Eyring,et al.  Evolving Obs4MIPs to Support Phase 6 of the Coupled Model Intercomparison Project (CMIP6) , 2015 .

[54]  Valerio Lucarini,et al.  New Results on the Thermodynamic Properties of the Climate System , 2011 .

[55]  F. Achard,et al.  Determination of tropical deforestation rates and related carbon losses from 1990 to 2010 , 2014, Global change biology.

[56]  Veronika Eyring,et al.  A community diagnostic tool for chemistry climate model validation , 2012 .

[57]  Jia Zong,et al.  Algorithm Theoretical Basis , 1999 .

[58]  Veronika Eyring,et al.  Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organization , 2015 .

[59]  François Massonnet,et al.  Quantifying climate feedbacks in polar regions , 2018, Nature Communications.

[60]  Clara Deser,et al.  Sea surface temperature variability: patterns and mechanisms. , 2010, Annual review of marine science.

[61]  Jana Sillmann,et al.  Extreme Cold Winter Temperatures in Europe under the Influence of North Atlantic Atmospheric Blocking , 2011 .

[62]  Michael Steele,et al.  PHC: A Global Ocean Hydrography with a High-Quality Arctic Ocean , 2001 .

[63]  R. Kwok.  Arctic sea ice thickness, volume, and multiyear ice coverage: losses and coupled variability (1958–2018) , 2018, Environmental Research Letters.

[64]  A. Phillips,et al.  Tropical Atlantic Influence on European Heat Waves , 2005 .

[65]  G. Meehl,et al.  The Coupled Model Intercomparison Project (CMIP) , 2000 .

[66]  D. A. Rothrock,et al.  Modeling Global Sea Ice with a Thickness and Enthalpy Distribution Model in Generalized Curvilinear Coordinates , 2003 .

[67]  P. Cox,et al.  Projected land photosynthesis constrained by changes in the seasonal cycle of atmospheric CO2 , 2016, Nature.

[68]  M. Holland,et al.  Trends in Arctic sea ice extent from CMIP5, CMIP3 and observations , 2012 .

[69]  P. Cox,et al.  Sensitivity of tropical carbon to climate change constrained by carbon dioxide variability , 2013, Nature.

[70]  Kevin E. Trenberth,et al.  Atlantic hurricanes and natural variability in 2005 , 2006 .

[71]  Goran Georgievski,et al.  Characterizing uncertainties in the ESA-CCI land cover map of the epoch 2010 and their impacts on MPI-ESM climate simulations , 2018, Theoretical and Applied Climatology.

[72]  C. Deser,et al.  Evaluating Modes of Variability in Climate Models , 2014 .

[73]  J. Norfolk,et al.  Primary Production of the Biosphere , 1976 .