Parametric Sensitivity and Uncertainty Quantification in the Version 1 of E3SM Atmosphere Model Based on Short Perturbed Parameter Ensemble Simulations

The atmospheric component of Energy Exascale Earth System Model (E3SM) version 1 (EAMv1) has included many new features in the physics parameterizations compared to its predecessors. Potential complex nonlinear interactions among the new features create a significant challenge for understanding the model behaviors and parameter tuning. Using the one-at-a-time method, the benefit of tuning one parameter may offset the benefit of tuning another parameter, or improvement in one target variable may lead to degradation in another target variable. To better understand the EAMv1 model behaviors and physics, we conducted a large number of short simulations (3 days) in which 18 parameters carefully selected from parameterizations of deep convection, shallow convection and cloud macrophysics and microphysics were perturbed simultaneously using the Latin Hypercube sampling method. From the Perturbed Parameters Ensemble (PPE) simulations and use of different skill score functions, we identified the most sensitive parameters, quantified how the model responds to changes of the parameters for both global mean and spatial distribution, and estimated the maximum likelihood of model parameter space for a number of important fidelity metrics. Comparison of the parametric sensitivity using simulations of two different lengths suggests that PPE using short simulations has some bearing on understanding parametric sensitivity of longer simulations. Results from this analysis provide a more comprehensive picture of the EAMv1 behavior. The difficulty in reducing biases in multiple variables simultaneously highlights the need of characterizing model structural uncertainty (so-called embedded errors) to inform future development efforts.

[1]  Soroosh Sorooshian,et al.  Exploring parameter sensitivities of the land surface using a locally coupled land-atmosphere model , 2004 .

[2]  Norman A. McFarlane,et al.  The Effect of Orographically Excited Gravity Wave Drag on the General Circulation of the Lower Stratosphere and Troposphere , 1987 .

[3]  Andrew Gettelman,et al.  The Art and Science of Climate Model Tuning , 2017 .

[4]  James C McWilliams,et al.  Considerations for parameter optimization and sensitivity in climate models , 2010, Proceedings of the National Academy of Sciences.

[5]  P. Thornton,et al.  The Impact of Parametric Uncertainties on Biogeochemistry in the E3SM Land Model , 2017 .

[6]  Yun Qian,et al.  Some issues in uncertainty quantification and parameter tuning: a case study of convective parameterization scheme in the WRF regional climate model , 2011 .

[7]  Jon C. Helton,et al.  Latin Hypercube Sampling and the Propagation of Uncertainty in Analyses of Complex Systems , 2002 .

[8]  R. Neale,et al.  Understanding Cloud and Convective Characteristics in Version 1 of the E3SM Atmosphere Model , 2018, Journal of Advances in Modeling Earth Systems.

[9]  Shaocheng Xie,et al.  On the Correspondence between Short- and Long-Time-Scale Systematic Errors in CAM4/CAM5 for the Year of Tropical Convection , 2012 .

[10]  Hui Wan,et al.  Short ensembles: an efficient method for discerning climate-relevant sensitivities in atmospheric general circulation models , 2014 .

[11]  Tianjun Zhou,et al.  Parameter Tuning and Calibration of RegCM3 with MIT–Emanuel Cumulus Parameterization Scheme over CORDEX East Asia Domain , 2014 .

[12]  Gabriel Huerta,et al.  Uncertainty Quantification in Climate Modeling and Projection , 2016 .

[13]  Damien Decremer,et al.  Strategies for reducing the climate noise in model simulations: ensemble runs versus a long continuous run , 2015, Climate Dynamics.

[14]  Jonathan Rougier,et al.  Analyzing the Climate Sensitivity of the HadSM3 Climate Model Using Ensembles from Different but Related Experiments , 2009 .

[15]  Vincent E. Larson,et al.  CLUBB-SILHS: A parameterization of subgrid variability in the atmosphere , 2017, 1711.03675.

[16]  D. Wallom,et al.  Climate model forecast biases assessed with a perturbed physics ensemble , 2017, Climate Dynamics.

[17]  A. OHagan,et al.  Bayesian analysis of computer code outputs: A tutorial , 2006, Reliab. Eng. Syst. Saf..

[18]  Xiaohong Liu,et al.  A new approach to modeling aerosol effects on East Asian climate: Parametric uncertainties associated with emissions, cloud microphysics, and their interactions , 2015 .

[19]  Faming Liang,et al.  Annealing evolutionary stochastic approximation Monte Carlo for global optimization , 2011, Stat. Comput..

[20]  Guang Lin,et al.  Sensitivity of surface flux simulations to hydrologic parameters based on an uncertainty quantification framework applied to the Community Land Model , 2012 .

[21]  Matthew D. Collins,et al.  UK Climate Projections Science Report: Climate Change Projections , 2009 .

[22]  Malte Prieß,et al.  Surrogate-based optimization of climate model parameters using response correction , 2011, J. Comput. Sci..

[23]  C. Hannay,et al.  The Role of Convective Gustiness in Reducing Seasonal Precipitation Biases in the Tropical West Pacific , 2018 .

[24]  S. Ghan,et al.  Parametric behaviors of CLUBB in simulations of low clouds in the Community Atmosphere Model (CAM) , 2014 .

[25]  Peter Challenor,et al.  The impact of structural error on parameter constraint in a climate model , 2016 .

[26]  W. Collins,et al.  The Community Earth System Model: A Framework for Collaborative Research , 2013 .

[27]  Mrinal K. Sen,et al.  An Efficient Stochastic Bayesian Approach to Optimal Parameter and Uncertainty Estimation for Climate Model Predictions , 2004 .

[28]  N. McFarlane,et al.  Sensitivity of Climate Simulations to the Parameterization of Cumulus Convection in the Canadian Climate Centre General Circulation Model , 1995, Data, Models and Analysis.

[29]  Vincent E. Larson,et al.  A sensitivity analysis of cloud properties to CLUBB parameters in the single‐column Community Atmosphere Model (SCAM5) , 2014 .

[30]  Vincent E. Larson,et al.  A PDF-Based Model for Boundary Layer Clouds. Part I: Method and Model Description , 2002 .

[31]  Wei Gong,et al.  An evaluation of adaptive surrogate modeling based optimization with two benchmark problems , 2014, Environ. Model. Softw..

[32]  David R. Doelling,et al.  Toward Optimal Closure of the Earth's Top-of-Atmosphere Radiation Budget , 2009 .

[33]  D. P. Schanen,et al.  Higher-Order Turbulence Closure and Its Impact on Climate Simulations in the Community Atmosphere Model , 2013 .

[34]  M. Webb,et al.  Climate model errors, feedbacks and forcings: a comparison of perturbed physics and multi-model ensembles , 2011 .

[35]  Andrew Gettelman,et al.  Advanced two-moment bulk microphysics for global models. Part I: off-line tests and comparison with other schemes. , 2015 .

[36]  R. Marchand,et al.  Hydrometeor Detection Using Cloudsat—An Earth-Orbiting 94-GHz Cloud Radar , 2008 .

[37]  M. Stein.  Large sample properties of simulations using latin hypercube sampling , 1987 .

[38]  Sally A. McFarlane,et al.  Uncertainty quantification and parameter tuning in the CAM5 Zhang‐McFarlane convection scheme and impact of improved convection on the global circulation and climate , 2012 .

[39]  J. Murphy,et al.  A methodology for probabilistic predictions of regional climate change from perturbed physics ensembles , 2007, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[40]  Derek J. Posselt,et al.  Linearization of Microphysical Parameterization Uncertainty Using Multiplicative Process Perturbation Parameters , 2014 .

[41]  Wei Gong,et al.  Assessing parameter importance of the Common Land Model based on qualitative and quantitative sensitivity analysis , 2013 .

[42]  K. Trenberth,et al.  Estimates of the Global Water Budget and Its Annual Cycle Using Observational and Model Data , 2007 .

[43]  D. Sexton,et al.  Climate projections of future extreme events accounting for modelling uncertainties and historical simulation biases , 2014, Climate Dynamics.

[44]  P. Xie,et al.  Global Precipitation: A 17-Year Monthly Analysis Based on Gauge Observations, Satellite Estimates, and Numerical Model Outputs , 1997 .

[45]  L. Ingber.  Very fast simulated re-annealing , 1989 .

[46]  Mrinal K. Sen,et al.  Error Reduction and Convergence in Climate Prediction , 2008 .

[47]  Derek J. Posselt,et al.  A Bayesian Examination of Deep Convective Squall-Line Sensitivity to Changes in Cloud Microphysical Parameters , 2016 .

[48]  Masahiro Watanabe,et al.  The Transpose-AMIP II Experiment and Its Application to the Understanding of Southern Ocean Cloud Biases in Climate Models , 2012 .

[49]  C. A. Severijns,et al.  Optimizing Parameters in an Atmospheric General Circulation Model , 2005 .

[50]  Heikki Haario,et al.  Optimization of NWP model closure parameters using total energy norm of forecast error as a target , 2014 .

[51]  Andrew D. Friend,et al.  Carbon and nitrogen cycle dynamics in the O‐CN land surface model: 1. Model description, site‐scale evaluation, and sensitivity to parameter estimates , 2010 .

[52]  Sandia Report,et al.  Sensitivity of Precipitation to Parameter Values in the Community Atmosphere Model Version 5 , 2014 .

[53]  Heikki Haario,et al.  Parameter variations in prediction skill optimization at ECMWF , 2013 .

[54]  Michael Goldstein,et al.  History matching for exploring and reducing climate model parameter space using observations and a large perturbed physics ensemble , 2013, Climate Dynamics.

[55]  S. Bony,et al.  The GCM‐Oriented CALIPSO Cloud Product (CALIPSO‐GOCCP) , 2010 .

[56]  D. Legates,et al.  Mean seasonal and spatial variability in gauge‐corrected, global precipitation , 1990 .

[57]  Jason Lowe,et al.  Transient climate changes in a perturbed parameter ensemble of emissions-driven earth system model simulations , 2014, Climate Dynamics.

[58]  L. Jaeger,et al.  Monatskarten des Niederschlags für die ganze Erde , 1976 .

[59]  Andrew Gettelman,et al.  Advanced Two-Moment Bulk Microphysics for Global Models. Part II: Global Model Solutions and Aerosol–Cloud Interactions* , 2015 .

[60]  Vincent E. Larson,et al.  PDF Parameterization of Boundary Layer Clouds in Models with Horizontal Grid Spacings from 2 to 16 km , 2012 .

[61]  Omar Bellprat,et al.  Objective calibration of regional climate models: OBJECTIVE CALIBRATION OF RCMS , 2012 .

[62]  Yu Sun,et al.  Inverse modeling of hydrologic parameters using surface flux and runoff observations in the Community Land Model , 2013 .

[63]  S. Bony,et al.  On the Correspondence between Mean Forecast Errors and Climate Errors in CMIP5 Models , 2013 .

[64]  Richard Neale,et al.  Parametric sensitivity analysis of precipitation at global and local scales in the Community Atmosphere Model CAM5 , 2015 .

[65]  F. Wentz,et al.  How Much More Rain Will Global Warming Bring? , 2007, Science.

[66]  James M. Salter,et al.  Identifying and removing structural biases in climate models with history matching , 2015, Climate Dynamics.

[67]  Yoram Rubin,et al.  On minimum relative entropy concepts and prior compatibility issues in vadose zone inverse and forward modeling , 2005 .

[68]  Leonard A. Smith,et al.  Uncertainty in predictions of the climate response to rising levels of greenhouse gases , 2005, Nature.

[69]  C. Chuang,et al.  An improved hindcast approach for evaluation and diagnosis of physical processes in global climate models , 2015 .

[70]  J. Thepaut,et al.  The ERA‐Interim reanalysis: configuration and performance of the data assimilation system , 2011 .

[71]  G. Stephens.  Cloud Feedbacks in the Climate System: A Critical Review , 2005 .

[72]  Richard P. Allan,et al.  Combining satellite data and models to estimate cloud radiative effect at the surface and in the atmosphere , 2011 .

[73]  Regional assessment of the parameter‐dependent performance of CAM4 in simulating tropical clouds , 2012 .

[74]  Daniel B. Williamson,et al.  Tuning without over-tuning: parametric uncertainty quantification for the NEMO ocean model , 2016 .

[75]  Dirk Notz,et al.  How well must climate models agree with observations? , 2015, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[76]  Wei Gong,et al.  Multi-objective parameter optimization of common land model using adaptive surrogate modeling , 2014 .

[77]  Moustafa T. Chahine,et al.  The hydrological cycle and its influence on climate , 1992, Nature.

[78]  S. Sorooshian,et al.  A Review of Global Precipitation Data Sets: Data Sources, Estimation, and Intercomparisons , 2018 .

[79]  S. Ghan,et al.  Description and evaluation of a new four-mode version of the Modal Aerosol Module (MAM4) within version 5.3 of the Community Atmosphere Model , 2015 .

[80]  Sally A. McFarlane,et al.  A Sensitivity Study of Radiative Fluxes at the Top of Atmosphere to Cloud-Microphysics and Aerosol Parameters in the Community Atmosphere Model CAM5 , 2013 .

[81]  B. Barkstrom,et al.  Earth Radiation Budget Experiment (ERBE): An Overview , 1982 .

[82]  V. Larson,et al.  Using Probability Density Functions to Derive Consistent Closure Relationships among Higher-Order Moments , 2005 .

[83]  Derek J. Posselt,et al.  Robust Characterization of Model Physics Uncertainty for Simulations of Deep Moist Convection , 2010 .

[84]  M. Kanamitsu,et al.  NCEP–DOE AMIP-II Reanalysis (R-2) , 2002 .

[85]  K. Trenberth,et al.  The Annual Cycle of the Energy Budget. Part I: Global Mean and Land–Ocean Exchanges , 2008 .

[86]  Vincent E. Larson,et al.  Elucidating Model Inadequacies in a Cloud Parameterization by Use of an Ensemble-Based Calibration Framework , 2007 .

[87]  J. Janowiak,et al.  The Version 2 Global Precipitation Climatology Project (GPCP) Monthly Precipitation Analysis (1979-Present) , 2003 .

[88]  Huiping Yan,et al.  Parametric sensitivity and calibration for the Kain-Fritsch convective parameterization scheme in the WRF model , 2014 .

[89]  Charles Doutriaux,et al.  A More Powerful Reality Test for Climate Models , 2016 .

[90]  Khachik Sargsyan,et al.  Bayesian Calibration of the Community Land Model Using Surrogates , 2012, SIAM/ASA J. Uncertain. Quantification.

[91]  Raphael T. Haftka,et al.  Surrogate-based Analysis and Optimization , 2005 .

[92]  Anthony O'Hagan,et al.  Diagnostics for Gaussian Process Emulators , 2009, Technometrics.

[93]  Derek J. Posselt,et al.  Quantification of Cloud Microphysical Parameterization Uncertainty using Radar Reflectivity , 2012 .

[94]  C. Covey,et al.  Efficient screening of climate model sensitivity to a large number of perturbed input parameters , 2013 .

[95]  David L. Williamson,et al.  Evaluating Parameterizations in General Circulation Models: Climate Simulation Meets Weather Prediction , 2004 .

[96]  K. Taylor.  Summarizing multiple aspects of model performance in a single diagram , 2001 .

[97]  Wei Gong,et al.  Assessing WRF model parameter sensitivity: A case study with 5 day summer precipitation forecasting in the Greater Beijing Area , 2015 .