Assessing parameter importance of the Common Land Model based on qualitative and quantitative sensitivity analysis

Abstract. Proper specification of model parameters is critical to the performance of land surface models (LSMs). Due to high dimensionality and parameter interaction, estimating parameters of an LSM is a challenging task. Sensitivity analysis (SA) is a tool that can screen out the most influential parameters on model outputs. In this study, we conducted parameter screening for six output fluxes for the Common Land Model: sensible heat, latent heat, upward longwave radiation, net radiation, soil temperature and soil moisture. A total of 40 adjustable parameters were considered. Five qualitative SA methods, including local, sum-of-trees, multivariate adaptive regression splines, delta test and Morris methods, were compared. The proper sampling design and sufficient sample size necessary to effectively screen out the sensitive parameters were examined. We found that there are 2–8 sensitive parameters, depending on the output type, and about 400 samples are adequate to reliably identify the most sensitive parameters. We also employed a revised Sobol' sensitivity method to quantify the importance of all parameters. The total effects of the parameters were used to assess the contribution of each parameter to the total variances of the model outputs. The results confirmed that global SA methods can generally identify the most sensitive parameters effectively, while local SA methods result in type I errors (i.e., sensitive parameters labeled as insensitive) or type II errors (i.e., insensitive parameters labeled as sensitive). Finally, we evaluated and confirmed the screening results for their consistency with the physical interpretation of the model parameters.

[1]  S. Sorooshian,et al.  Calibration of watershed models , 2003 .

[2]  John C. Schaake,et al.  A Priori Estimation of Land Surface Model Parameters , 2013 .

[3]  Joseph A. C. Delaney.  Sensitivity analysis , 2018, The African Continental Free Trade Area: Economic and Distributional Effects.

[4]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[5]  Michel Verleysen,et al.  Using the Delta Test for Variable Selection , 2008, ESANN.

[6]  S. Sorooshian,et al.  Shuffled complex evolution approach for effective and efficient global minimization , 1993 .

[7]  B. Bonan,et al.  A Land Surface Model (LSM Version 1.0) for Ecological, Hydrological, and Atmospheric Studies: Technical Description and User's Guide , 1996 .

[8]  H Christopher Frey,et al.  Comparison of Sensitivity Analysis Methods Based on Applications to a Food Safety Risk Assessment Model , 2004, Risk analysis : an official publication of the Society for Risk Analysis.

[9]  Roman B. Statnikov,et al.  MOVI 1.0 Operation , 2002 .

[10]  G. Fu,et al.  Sobol′’s sensitivity analysis for a distributed hydrological model of Yichun River Basin, China , 2013 .

[11]  D. Vries.  Thermal properties of soils , 1963 .

[12]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[13]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[14]  Soroosh Sorooshian,et al.  Sensitivity analysis of a land surface scheme using multicriteria methods , 1999 .

[15]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[16]  T. Turányi.  Sensitivity analysis of complex kinetic systems. Tools and applications , 1990 .

[17]  Jon C. Helton,et al.  Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models , 2009, Reliab. Eng. Syst. Saf..

[18]  Marc B Neumann,et al.  Comparison of sensitivity analysis methods for pollutant degradation modelling: a case study from drinking water treatment. , 2012, The Science of the total environment.

[19]  R. Dickinson,et al.  The Common Land Model , 2003 .

[20]  X. Y. Sun,et al.  Three complementary methods for sensitivity analysis of a water quality model , 2012, Environ. Model. Softw..

[21]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[22]  Tiranee Achalakul,et al.  Yield improvement analysis with parameter-screening factorials , 2012, Appl. Soft Comput..

[23]  Keith Beven,et al.  Bayesian estimation of uncertainty in land surface‐atmosphere flux predictions , 1997 .

[24]  George R. Blake,et al.  Thermal Properties of Soils , 1950 .

[25]  Carolina Massmann,et al.  Analysis of the behavior of a rainfall-runoff model using three global sensitivity analysis methods evaluated at different temporal scales , 2012 .

[26]  S. Pandis,et al.  Dimethylsulfide chemistry in the remote marine atmosphere: Evaluation and sensitivity analysis of available mechanisms , 1997 .

[27]  Stefano Tarantola,et al.  Evaluation of MARS modeling technique for sensitivity analysis of model output , 2010 .

[28]  Zeng Qingcun,et al.  A land surface model (IAP94) for climate studies part I: Formulation and validation in off-line experiments , 1997 .

[29]  Stefano Tarantola,et al.  Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models , 2004 .

[30]  G. Hornberger,et al.  A Statistical Exploration of the Relationships of Soil Moisture Characteristics to the Physical Properties of Soils , 1984 .

[31]  Holger R. Maier,et al.  A comparison of sensitivity analysis techniques for complex models for environmental management , 2005 .

[32]  Carsten Peterson,et al.  Finding the Embedding Dimension and Variable Dependencies in Time Series , 1994, Neural Computation.

[33]  DebK.,et al.  A fast and elitist multiobjective genetic algorithm , 2002 .

[34]  A. Saltelli,et al.  Importance measures in global sensitivity analysis of nonlinear models , 1996 .

[35]  Gianni Bellocchi,et al.  Comparison of sensitivity analysis techniques: A case study with the rice model WARM , 2010 .

[36]  P. Reed,et al.  Hydrology and Earth System Sciences Discussions Comparing Sensitivity Analysis Methods to Advance Lumped Watershed Model Identification and Evaluation , 2022 .

[37]  F. Graziani,et al.  A Practical Global Sensitivity Analysis Methodology for Multi-Physics Applications , 2008 .

[38]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[39]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[40]  Ignacio Rojas,et al.  Minimising the delta test for variable selection in regression problems , 2008, Int. J. High Perform. Syst. Archit..

[41]  Max D. Morris,et al.  Factorial sampling plans for preliminary computational experiments , 1991 .

[42]  W. James Shuttleworth,et al.  A fully multiple-criteria implementation of the Sobol' method for parameter sensitivity analysis , 2012 .

[43]  Q. Duana,et al.  Model Parameter Estimation Experiment (MOPEX): An overview of science strategy and major results from the second and third workshops , 2006 .

[44]  Florian Pappenberger,et al.  Multi-method global sensitivity analysis of flood inundation models. , 2008 .

[45]  Willy Bauwens,et al.  Sobol' sensitivity analysis of a complex environmental model , 2011, Environ. Model. Softw..

[46]  Ann Henderson-Sellers,et al.  Soil moisture: A critical focus for global change studies , 1996 .

[47]  W. J. Shuttleworth,et al.  Parameter estimation of a land surface scheme using multicriteria methods , 1999 .

[48]  Cajo J. F. ter Braak,et al.  Treatment of input uncertainty in hydrologic modeling: Doing hydrology backward with Markov chain Monte Carlo simulation , 2008 .

[49]  R. Jordan.  A One-dimensional temperature model for a snow cover : technical documentation for SNTHERM.89 , 1991 .

[50]  A. Saltelli,et al.  Sensitivity analysis of an environmental model: an application of different analysis methods , 1997 .

[51]  Richard J. Beckman,et al.  A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output From a Computer Code , 2000, Technometrics.

[52]  E. Anderson,et al.  A point energy and mass balance model of a snow cover , 1975 .

[53]  Roni Avissar,et al.  An Evaluation with the Fourier Amplitude Sensitivity Test (FAST) of Which Land-Surface Parameters Are of Greatest Importance in Atmospheric Modeling , 1994 .

[54]  Saltelli Andrea,et al.  Sensitivity Analysis for Nonlinear Mathematical Models. Numerical ExperienceSensitivity Analysis for Nonlinear Mathematical Models. Numerical Experience , 1995 .

[55]  Andrea Saltelli,et al.  An effective screening design for sensitivity analysis of large models , 2007, Environ. Model. Softw..

[56]  Mrinal K. Sen,et al.  Optimal parameter and uncertainty estimation of a land surface model: A case study using data from Cabauw, Netherlands , 2003 .

[57]  Patrick M. Reed,et al.  Technical Note: Method of Morris effectively reduces the computational demands of global sensitivity analysis for distributed watershed models , 2013 .

[58]  Jerome H. Friedman,et al.  Rejoinder: Multivariate Adaptive Regression Splines , 1991 .

[59]  Roman B. Statnikov,et al.  Multicriteria Analysis in Engineering: Using the PSI Method with MOVI 1.0 , 2002 .

[60]  Bernard De Baets,et al.  Unravelling the output uncertainty of a tree water flow and storage model using several global sensitivity analysis methods , 2008 .

[61]  G. Hornberger,et al.  Empirical equations for some soil hydraulic properties , 1978 .

[62]  I. Sobol.  Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates , 2001 .

[63]  Xin-Zhong Liang,et al.  Evaluation of a Conjunctive Surface–Subsurface Process Model (CSSP) over the Contiguous United States at Regional–Local Scales , 2011 .

[64]  H. Chipman,et al.  BART: Bayesian Additive Regression Trees , 2008, 0806.3286.

[65]  Soroosh Sorooshian,et al.  Constraining Land Surface and Atmospheric Parameters of a Locally Coupled Model Using Observational Data , 2005 .

[66]  H. Chipman,et al.  Bayesian Additive Regression Trees , 2006 .

[67]  Guang Lin,et al.  Sensitivity of surface flux simulations to hydrologic parameters based on an uncertainty quantification framework applied to the Community Land Model , 2012 .

[68]  Piers J. Sellers,et al.  A Global Climatology of Albedo, Roughness Length and Stomatal Resistance for Atmospheric General Circulation Models as Represented by the Simple Biosphere Model (SiB) , 1989 .

[69]  Xu Liang,et al.  Multi-Resolution Calibration Methodology for Hydrologic Models: Application to a Sub-Humid Catchment , 2013 .

[70]  Saltelli Andrea,et al.  Global Sensitivity Analysis: The Primer , 2008 .

[71]  J. Freidman,et al.  Multivariate adaptive regression splines , 1991 .