What do we mean by sensitivity analysis? The need for comprehensive characterization of “global” sensitivity in Earth and Environmental systems models

Sensitivity analysis is an essential paradigm in Earth and Environmental Systems modeling. However, the term “sensitivity” has a clear definition, based in partial derivatives, only when specified locally around a particular point (e.g., optimal solution) in the problem space. Accordingly, no unique definition exists for “global sensitivity” across the problem space, when considering one or more model responses to different factors such as model parameters or forcings. A variety of approaches have been proposed for global sensitivity analysis, based on different philosophies and theories, and each of these formally characterizes a different “intuitive” understanding of sensitivity. These approaches focus on different properties of the model response at a fundamental level and may therefore lead to different (even conflicting) conclusions about the underlying sensitivities. Here we revisit the theoretical basis for sensitivity analysis, summarize and critically evaluate existing approaches in the literature, and demonstrate their flaws and shortcomings through conceptual examples. We also demonstrate the difficulty involved in interpreting “global” interaction effects, which may undermine the value of existing interpretive approaches. With this background, we identify several important properties of response surfaces that are associated with the understanding and interpretation of sensitivities in the context of Earth and Environmental System models. Finally, we highlight the need for a new, comprehensive framework for sensitivity analysis that effectively characterizes all of the important sensitivity-related properties of model response surfaces.

[1]  C. Fortuin,et al.  Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. I Theory , 1973 .

[2]  B. Croke,et al.  Addressing ten questions about conceptual rainfall–runoff models with global sensitivity analyses in R , 2013 .

[3]  R Fisher,et al.  Design of Experiments , 1936 .

[4]  Douglas C. Montgomery,et al.  Response Surface Methodology: Process and Product Optimization Using Designed Experiments , 1995 .

[5]  T. Brubaker,et al.  Nonlinear Parameter Estimation , 1979 .

[6]  M. Kramer,et al.  Sensitivity Analysis in Chemical Kinetics , 1983 .

[7]  R. Srinivasan,et al.  A global sensitivity analysis tool for the parameters of multi-variable catchment models , 2006 .

[8]  R. Iman,et al.  The Use of the Rank Transform in Regression , 1979 .

[9]  R. Lidén,et al.  Analysis of conceptual rainfall–runoff modelling performance in different climates , 2000 .

[10]  R. Spear Eutrophication in peel inlet—II. Identification of critical uncertainties via generalized sensitivity analysis , 1980 .

[11]  D. Hamby A review of techniques for parameter sensitivity analysis of environmental models , 1994, Environmental monitoring and assessment.

[12]  Nong Shang,et al.  Parameter uncertainty and interaction in complex environmental models , 1994 .

[13]  Stefano Tarantola,et al.  Sensitivity and uncertainty analysis in spatial modelling based on GIS , 2000 .

[14]  Willy Bauwens,et al.  Sobol' sensitivity analysis of a complex environmental model , 2011, Environ. Model. Softw..

[15]  W. J. Shuttleworth,et al.  Parameter estimation of a land surface scheme using multicriteria methods , 1999 .

[16]  Yuqiong Liu,et al.  Reconciling theory with observations: elements of a diagnostic approach to model evaluation , 2008 .

[17]  Saltelli Andrea,et al.  Global Sensitivity Analysis: The Primer , 2008 .

[18]  Ralf Takors,et al.  Sensitivity analysis for the reduction of complex metabolism models , 2004 .

[19]  A. Saltelli,et al.  A quantitative model-independent method for global sensitivity analysis of model output , 1999 .

[20]  Sergei S. Kucherenko,et al.  Derivative based global sensitivity measures and their link with global sensitivity indices , 2009, Math. Comput. Simul..

[21]  Stefano Tarantola,et al.  Sensitivity analysis in model calibration: GSA-GLUE approach , 2001 .

[22]  Soroosh Sorooshian,et al.  Parameter sensitivity analysis for different complexity land surface models using multicriteria methods , 2006 .

[23]  Paola Annoni,et al.  Sixth International Conference on Sensitivity Analysis of Model Output How to avoid a perfunctory sensitivity analysis , 2010 .

[24]  A. Saltelli,et al.  An alternative way to compute Fourier amplitude sensitivity test (FAST) , 1998 .

[25]  Dmitri Kavetski,et al.  Pursuing the method of multiple working hypotheses for hydrological modeling , 2011 .

[26]  I. Sobola,et al.  Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates , 2001 .

[27]  S. Sorooshian,et al.  Effective and efficient global optimization for conceptual rainfall‐runoff models , 1992 .

[28]  A. W. F. Edwards,et al.  Statistical Inference. (Book Reviews: Likelihood. An Account of the Statistical Concept of Likelihood and Its Application to Scientific Inference) , 1973 .

[29]  Bryan A. Tolson,et al.  Numerical assessment of metamodelling strategies in computationally intensive optimization , 2012, Environ. Model. Softw..

[30]  Hoshin Vijai Gupta,et al.  Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling , 2009 .

[31]  A. Saltelli,et al.  Importance measures in global sensitivity analysis of nonlinear models , 1996 .

[32]  Bryan A. Tolson,et al.  Review of surrogate modeling in water resources , 2012 .

[33]  Thomas J. Santner,et al.  The Design and Analysis of Computer Experiments , 2003, Springer Series in Statistics.

[34]  Stefano Tarantola,et al.  Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models , 2004 .

[35]  Florian Pappenberger,et al.  Multi-method global sensitivity analysis of flood inundation models. , 2008 .

[36]  Mary C. Hill,et al.  Distributed Evaluation of Local Sensitivity Analysis (DELSA), with application to hydrologic models , 2014 .

[37]  Soroosh Sorooshian,et al.  Toward improved calibration of hydrologic models: Multiple and noncommensurable measures of information , 1998 .

[38]  Alain Pietroniro,et al.  Development of the MESH modelling system for hydrological ensemble forecasting of the Laurentian Great Lakes at the regional scale , 2006 .

[39]  S. Sorooshian,et al.  A Shuffled Complex Evolution Metropolis algorithm for optimization and uncertainty assessment of hydrologic model parameters , 2002 .

[40]  M. Clark,et al.  Ancient numerical daemons of conceptual hydrological modeling: 2. Impact of time stepping schemes on model analysis and prediction , 2010 .

[41]  Max D. Morris,et al.  Factorial sampling plans for preliminary computational experiments , 1991 .

[42]  George Kuczera,et al.  Monte Carlo assessment of parameter uncertainty in conceptual catchment models: the Metropolis algorithm , 1998 .

[43]  W. James Shuttleworth,et al.  A fully multiple-criteria implementation of the Sobol' method for parameter sensitivity analysis , 2012 .

[44]  Agus Sudjianto,et al.  Relative Entropy Based Method for Probabilistic Sensitivity Analysis in Engineering Design , 2006 .

[45]  Igor Linkov,et al.  Exploring vulnerability of coastal habitats to sea level rise through global sensitivity and uncertainty analyses , 2011, Environ. Model. Softw..

[46]  Ilya M. Sobol,et al.  Sensitivity Estimates for Nonlinear Mathematical Models , 1993 .

[47]  Misgana K. Muleta,et al.  Sensitivity and uncertainty analysis coupled with automatic calibration for a distributed watershed model , 2005 .

[48]  José Ferrer,et al.  An improved sampling strategy based on trajectory design for application of the Morris method to systems with many input factors , 2012, Environ. Model. Softw..

[49]  Jack P. C. Kleijnen,et al.  Sensitivity analysis and optimization of system dynamics models: Regression analysis and statistical design of experiments , 1995 .

[50]  Andrea Saltelli,et al.  An effective screening design for sensitivity analysis of large models , 2007, Environ. Model. Softw..

[51]  Jack P. C. Kleijnen,et al.  An Overview of the Design and Analysis of Simulation Experiments for Sensitivity Analysis , 2005, Eur. J. Oper. Res..