Process-Oriented Evaluation of Climate and Weather Forecasting Models

Realistic climate and weather prediction models are necessary to produce confidence in projections of future climate over many decades and predictions for days to seasons. These models must be physically justified and validated for multiple weather and climate processes. A key opportunity to accelerate model improvement is greater incorporation of process-oriented diagnostics (PODs) into standard packages that can be applied during the model development process, allowing the application of diagnostics to be repeatable across multiple model versions and used as a benchmark for model improvement. A POD characterizes a specific physical process or emergent behavior that is related to the ability to simulate an observed phenomenon. This paper describes the outcomes of activities by the Model Diagnostics Task Force (MDTF) under the NOAA Climate Program Office (CPO) Modeling, Analysis, Predictions and Projections (MAPP) program to promote development of PODs and their application to climate and weather prediction models. MDTF and modeling center perspectives on the need for expanded process-oriented diagnosis of models are presented. Multiple PODs developed by the MDTF are summarized, and an open-source software framework developed by the MDTF to aid application of PODs to centers’ model development is presented in the context of other relevant community activities. The paper closes by discussing paths forward for the MDTF effort and for community process-oriented diagnosis.

[1]  C. Naud,et al.  Extratropical Cyclone Clouds in the GFDL Climate Model: Diagnosing Biases and the Associated Causes , 2019, Journal of Climate.

[2]  G. Vecchi,et al.  Moist Static Energy Budget Analysis of Tropical Cyclone Intensification in High-Resolution Climate Models , 2019, Journal of Climate.

[3]  Bettina K. Gier,et al.  Taking climate model evaluation to the next level , 2019, Nature Climate Change.

[4]  R. Small,et al.  Atmosphere surface storm track response to resolved ocean mesoscale in two sets of global climate model experiments , 2019, Climate Dynamics.

[5]  Randal D. Koster,et al.  GLACE: The Global Land-Atmosphere Coupling Experiment , 2018 .

[6]  B. Brown,et al.  Statistics of multi‐year droughts from the method for object‐based diagnostic evaluation , 2018 .

[7]  A. Berg,et al.  Soil Moisture–Evapotranspiration Coupling in CMIP5 Models: Relationship with Simulated Climate and Projections , 2018, Journal of Climate.

[8]  J. David Neelin,et al.  Convective Transition Statistics over Tropical Oceans for Climate Model Diagnostics: Observational Baseline , 2018 .

[9]  S. Giangrande,et al.  GoAmazon2014/5 campaign points to deep-inflow approach to deep convection across scales , 2018, Proceedings of the National Academy of Sciences.

[10]  E. Barnes,et al.  Memory Matters: A Case for Granger Causality in Climate Variability Studies , 2018 .

[11]  J. R. Wilson,et al.  The GFDL Global Atmosphere and Land Model AM4.0/LM4.0: 1. Simulation Characteristics With Prescribed SSTs , 2018 .

[12]  J. R. Wilson,et al.  The GFDL Global Atmosphere and Land Model AM4.0/LM4.0: 2. Model Description, Sensitivity Studies, and Tuning Strategies , 2018 .

[13]  E. Barnes,et al.  Skillful empirical subseasonal prediction of landfalling atmospheric river activity using the Madden–Julian oscillation and quasi-biennial oscillation , 2018, npj Climate and Atmospheric Science.

[14]  C. Naud,et al.  Evaluation of Extratropical Cyclone Precipitation in the North Atlantic Basin: An analysis of ERA-Interim, WRF, and two CMIP5 models. , 2017, Journal of climate.

[15]  Duane E. Waliser,et al.  A Unified Moisture Mode Framework for Seasonality of the Madden–Julian Oscillation , 2017, Journal of Climate.

[16]  Yilin Lu.  Surface Air Temperature Biases and Other Associated Biases in GFDL AM4.0 Model , 2018 .

[17]  G. Vecchi,et al.  Process-oriented diagnosis of tropical cyclones in high-resolution GCMs , 2017 .

[18]  Stephen A. Klein,et al.  The Cloud Feedback Model Intercomparison Project (CFMIP) Diagnostic Codes Catalogue – metrics, diagnostics and methodologies to evaluate, understand and improve the representation of clouds and cloud feedbacks in climate models , 2017 .

[19]  J. Mülmenstädt,et al.  A Multimodel Study on Warm Precipitation Biases in Global Models Compared to Satellite Observations , 2017 .

[20]  B. Taguchi,et al.  Systematic Errors in South Asian Monsoon Simulation: Importance of Equatorial Indian Ocean Processes , 2017 .

[21]  E. Barnes,et al.  Advancing atmospheric river forecasts into subseasonal‐to‐seasonal time scales , 2017 .

[22]  R. Small,et al.  Spatial patterns and intensity of the surface storm tracks in CMIP5 models , 2017 .

[23]  E. Maloney,et al.  Madden–Julian Oscillation Pacific Teleconnections: The Impact of the Basic State and MJO Representation in General Circulation Models , 2017 .

[24]  E. Maloney,et al.  MJO simulation in CMIP5 climate models: MJO skill metrics and process-oriented diagnosis , 2017, Climate Dynamics.

[25]  Daehyun Kim,et al.  Why does the MJO detour the Maritime Continent during austral summer? , 2017 .

[26]  Winter mean lower tropospheric moisture over the Maritime Continent as a climate model diagnostic metric for the propagation of the Madden‐Julian oscillation , 2017 .

[27]  C. Roberto Mechoso,et al.  Tropical Convective Transition Statistics and Causality in the Water Vapor–Precipitation Relation , 2017 .

[28]  Xianan Jiang,et al.  Key processes for the eastward propagation of the Madden‐Julian Oscillation based on multimodel simulations , 2017 .

[29]  B. Brown,et al.  Evaluating NARCCAP model performance for frequencies of severe-storm environments , 2016 .

[30]  Eric Guilyardi,et al.  Towards improved and more routine Earth system model evaluation in CMIP , 2016 .

[31]  A. D. Del Genio,et al.  The relationship between boundary layer stability and cloud cover in the post-cold frontal region. , 2016, Journal of climate.

[32]  E. Maloney,et al.  Convective moisture adjustment time scale as a key factor in regulating model amplitude of the Madden‐Julian Oscillation , 2016 .

[33]  E. Chassignet,et al.  Temperature–Salinity Structure of the North Atlantic Circulation and Associated Heat and Freshwater Transports , 2016 .

[34]  J. Neelin,et al.  Deep Convection and Column Water Vapor over Tropical Land versus Tropical Ocean: A Comparison between the Amazon and the Tropical Western Pacific , 2016 .

[35]  E. Barnes,et al.  The Influence of the Madden–Julian Oscillation on Northern Hemisphere Winter Blocking , 2016 .

[36]  Charles Doutriaux,et al.  A More Powerful Reality Test for Climate Models , 2016 .

[37]  A. Bodas‐Salcedo,et al.  Evaluation of the Warm Rain Formation Process in Global Models with Satellite Observations , 2015 .

[38]  John M. Wallace,et al.  Three-Dimensional Structure and Evolution of the Moisture Field in the MJO , 2015 .

[39]  Simon Read,et al.  ESMValTool (v1.0) – a community diagnostic and performance metrics tool for routine evaluation of Earth system models in CMIP , 2015 .

[40]  A. Genio,et al.  Role of Longwave Cloud-Radiation Feedback in the Simulation of the Madden-Julian Oscillation , 2015 .

[41]  Daehyun Kim,et al.  Vertical structure and physical processes of the Madden‐Julian oscillation: Exploring key model physics in climate simulations , 2015 .

[42]  Deep Convective Transition Characteristics in the Community Climate System Model and Changes under Global Warming , 2014 .

[43]  A. Sobel,et al.  Moist Static Energy Budget of the MJO during DYNAMO , 2014 .

[44]  Jaclyn N. Brown,et al.  Assessment of the CMIP5 global climate model simulations of the western tropical Pacific climate system and comparison to CMIP3 , 2014 .

[45]  E. Maloney,et al.  Gross Moist Stability and MJO Simulation Skill in Three Full-Physics GCMs , 2014 .

[46]  Richard Neale,et al.  Process-Oriented MJO Simulation Diagnostic: Moisture Sensitivity of Simulated Convection , 2014 .

[47]  E. Maloney,et al.  The moist static energy budget in NCAR CAM5 hindcasts during DYNAMO , 2014 .

[48]  Arun Kumar,et al.  A Framework for Dynamical Seasonal Prediction of Precipitation over the Pacific Islands , 2014 .

[49]  C. Mechoso,et al.  A global perspective on CMIP5 climate model biases , 2014 .

[50]  C. Naud,et al.  Evaluation of ERA-Interim and MERRA Cloudiness in the Southern Ocean , 2014 .

[51]  S. Xie,et al.  Tropical Biases in CMIP5 Multimodel Ensemble: The Excessive Equatorial Pacific Cold Tongue and Double ITCZ Problems* , 2014 .

[52]  M. Chikira.  Eastward-Propagating Intraseasonal Oscillation Represented by Chikira–Sugiyama Cumulus Parameterization. Part II: Understanding Moisture Variation under Weak Temperature Gradient Balance , 2014 .

[53]  A. Sobel,et al.  Propagating versus Nonpropagating Madden–Julian Oscillation Events , 2014 .

[54]  Kristopher,et al.  North American Climate in CMIP5 Experiments. Part II: Evaluation of Historical Simulations of Intraseasonal to Decadal Variability , 2013 .

[55]  A. Mariotti,et al.  North American Climate in CMIP5 experiments. Part I: Evaluation of historical simulations of continental and regional climatology , 2013 .

[56]  Dongxiao Zhang,et al.  Atlantic Meridional Overturning Circulation (AMOC) in CMIP5 Models: RCP and Historical Simulations , 2013 .

[57]  Jean-Christophe Golaz,et al.  Evaluating cloud tuning in a climate model with satellite observations , 2013 .

[58]  Daehyun Kim,et al.  MJO and Convectively Coupled Equatorial Waves Simulated by CMIP5 Climate Models , 2013 .

[59]  D. Neelin,et al.  California Winter Precipitation Change under Global Warming in the Coupled Model Intercomparison Project Phase 5 Ensemble , 2013 .

[60]  Kevin I. Hodges,et al.  The Ability of CMIP5 Models to Simulate North Atlantic Extratropical Cyclones , 2013 .

[61]  Kerry A. Emanuel,et al.  Physical mechanisms controlling self‐aggregation of convection in idealized numerical modeling simulations , 2013 .

[62]  S. Bony,et al.  What Are Climate Models Missing? , 2013, Science.

[63]  E. Guilyardi,et al.  ENSO representation in climate models: from CMIP3 to CMIP5 , 2013, Climate Dynamics.

[64]  Eric D. Maloney,et al.  Moisture Modes and the Eastward Propagation of the MJO , 2013 .

[65]  E. Maloney,et al.  Madden-Julian Oscillation Task Force : a joint effort of the climate and weather communities , 2013 .

[66]  Yaxing Wei,et al.  UV-CDAT: Analyzing Climate Datasets from a User's Perspective , 2013, Computing in Science & Engineering.

[67]  Lamont-Doherty Earth Observatory.  Propagating versus Nonpropagating Madden – Julian Oscillation Events , 2013 .

[68]  C. Tebaldi,et al.  Long-term Climate Change: Projections, Commitments and Irreversibility , 2013 .

[69]  Bin Wang,et al.  The Asian summer monsoon: an intercomparison of CMIP5 vs. CMIP3 simulations of the late 20th century , 2013, Climate Dynamics.

[70]  Philippe Ciais,et al.  A framework for benchmarking land models , 2012 .

[71]  Veronika Eyring,et al.  A community diagnostic tool for chemistry climate model validation , 2012 .

[72]  Sarah C. Jones,et al.  The ``Year'' of Tropical Convection (May 2008-April 2010): Climate Variability and Weather Highlights , 2012 .

[73]  A. Hall,et al.  California winter precipitation change under global warming in the Coupled Model 1 Intercomparison Project 5 ensemble 2 , 2012 .

[74]  Surendra Byna,et al.  TECA: A Parallel Toolkit for Extreme Climate Analysis , 2012, ICCS.

[75]  Richard Neale,et al.  Temperature–Moisture Dependence of the Deep Convective Transition as a Constraint on Entrainment in Climate Models , 2012 .

[76]  E. Maloney,et al.  Intraseasonal moist static energy budget in reanalysis data , 2011 .

[77]  E. Maloney,et al.  A Systematic Relationship between Intraseasonal Variability and Mean State Bias in AGCM Simulations , 2011 .

[78]  Aurel F. Moise,et al.  New climate model metrics based on object‐orientated pattern matching of rainfall , 2011 .

[79]  J. Thepaut,et al.  The ERA‐Interim reanalysis: configuration and performance of the data assimilation system , 2011 .

[80]  Richard Neale,et al.  Parameterizing Convective Organization to Escape the Entrainment Dilemma , 2011 .

[81]  Takashi Nakajima,et al.  Particle Growth and Drop Collection Efficiency of Warm Clouds as Inferred from Joint CloudSat and MODIS Observations , 2010 .

[82]  Takashi Nakajima,et al.  Droplet Growth in Warm Water Clouds Observed by the A-Train. Part II: A Multisensor View , 2010 .

[83]  S. Seneviratne,et al.  Investigating soil moisture-climate interactions in a changing climate: A review , 2010 .

[84]  J. David Neelin,et al.  The Transition to Strong Convection , 2009 .

[85]  J. David Neelin,et al.  Moisture Vertical Structure, Column Water Vapor, and Tropical Deep Convection , 2009 .

[86]  Bin Wang,et al.  MJO Simulation Diagnostics , 2009 .

[87]  Eric D. Maloney,et al.  The Moist Static Energy Budget of a Composite Tropical Intraseasonal Oscillation in a Climate Model , 2009 .

[88]  Susan C. van den Heever,et al.  Radiative–Convective Feedbacks in Idealized States of Radiative–Convective Equilibrium , 2008 .

[89]  R. Marchand,et al.  Hydrometeor Detection Using Cloudsat—An Earth-Orbiting 94-GHz Cloud Radar , 2008 .

[90]  Timothy M. Lenton,et al.  A wavelet‐coefficient score for comparison of two‐dimensional climatic‐data fields , 2008 .

[91]  David S. Nolan,et al.  Tropical Cyclone Intensification from Asymmetric Convection: Energetics and Efficiency , 2007 .

[92]  Randal D. Koster,et al.  Do Global Models Properly Represent the Feedback between Land and Atmosphere? , 2006, Journal of Hydrometeorology.

[93]  Matthew E. Peters,et al.  Interpretation of simple and cloud-resolving simulations of moist convection–radiation interaction with a mock-Walker circulation , 2006 .

[94]  Taikan Oki,et al.  GLACE: The Global Land–Atmosphere Coupling Experiment. Part II: Analysis , 2006, Journal of Hydrometeorology.

[95]  J. Neelin,et al.  Critical phenomena in atmospheric precipitation , 2006, cond-mat/0606076.

[96]  Matthew E. Peters,et al.  Relationships between Water Vapor Path and Precipitation over the Tropical Oceans , 2004 .

[97]  K. Trenberth,et al.  The Diurnal Cycle and Its Depiction in the Community Climate System Model , 2004 .

[98]  W. Paul Menzel,et al.  The MODIS cloud products: algorithms and examples from Terra , 2003, IEEE Trans. Geosci. Remote. Sens..

[99]  I. Held,et al.  Modeling Tropical Convergence Based on the Moist Static Energy Budget , 1987 .

[100]  James J. Hack,et al.  Inertial Stability and Tropical Cyclone Development , 1982 .

[101]  Brian J. Hoskins,et al.  The Steady Linear Response of a Spherical Atmosphere to Thermal and Orographic Forcing , 1981 .

[102]  John M. Wallace,et al.  Planetary-Scale Atmospheric Phenomena Associated with the Southern Oscillation , 1981 .