Insights into sensitivity analysis of Earth and environmental systems models: On the impact of parameter perturbation scale

Abstract This paper investigates the commonly overlooked “sensitivity” of sensitivity analysis (SA) to what we refer to as parameter “perturbation scale”, which can be defined as a prescribed size of the sensitivity-related neighbourhood around any point in the parameter space (analogous to step size Δ x for numerical estimation of derivatives). We discuss that perturbation scale is inherent to any (local and global) SA approach, and explain how derivative-based SA approaches (e.g., method of Morris) focus on small-scale perturbations, while variance-based approaches (e.g., method of Sobol) focus on large-scale perturbations. We employ a novel variogram-based approach, called Variogram Analysis of Response Surfaces (VARS), which bridges derivative- and variance-based approaches. Our analyses with different real-world environmental models demonstrate significant implications of subjectivity in the perturbation-scale choice and the need for strategies to address these implications. It is further shown how VARS can uniquely characterize the perturbation-scale dependency and generate sensitivity measures that encompass all sensitivity-related information across the full spectrum of perturbation scales.

[1]  Saman Razavi,et al.  Challenges and Future Outlook of Sensitivity Analysis , 2017 .

[2]  Paola Annoni,et al.  Sixth International Conference on Sensitivity Analysis of Model Output How to avoid a perfunctory sensitivity analysis , 2010 .

[3]  Saltelli Andrea,et al.  Global Sensitivity Analysis: The Primer , 2008 .

[4]  Peter Z. G. Qian.  Sliced Latin Hypercube Designs , 2012 .

[5]  Alain Pietroniro,et al.  Development of the MESH modelling system for hydrological ensemble forecasting of the Laurentian Great Lakes at the regional scale , 2006 .

[6]  Lauren M. Fry,et al.  The Great Lakes Runoff Intercomparison Project Phase 1: Lake Michigan (GRIP-M) , 2014 .

[7]  Saman Razavi,et al.  Progressive Latin Hypercube Sampling: An efficient approach for robust sampling-based analysis of environmental models , 2017, Environ. Model. Softw..

[8]  Soroosh Sorooshian,et al.  Sensitivity analysis of a land surface scheme using multicriteria methods , 1999 .

[9]  Marco Ratto,et al.  Global Sensitivity Analysis , 2008 .

[10]  Soroosh Sorooshian,et al.  Exploring parameter sensitivities of the land surface using a locally coupled land-atmosphere model , 2004 .

[11]  Andrea Saltelli,et al.  An effective screening design for sensitivity analysis of large models , 2007, Environ. Model. Softw..

[12]  L. Shawn Matott,et al.  Calibrating Environment Canada's MESH Modelling System over the Great Lakes Basin , 2014 .

[13]  H. Gupta,et al.  A new framework for comprehensive, robust, and efficient global sensitivity analysis: 2. Application , 2016 .

[14]  Jeffrey G. Arnold,et al.  Model Evaluation Guidelines for Systematic Quantification of Accuracy in Watershed Simulations , 2007 .

[15]  H. Gupta,et al.  A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory , 2016 .

[16]  Sergei S. Kucherenko,et al.  Derivative based global sensitivity measures and their link with global sensitivity indices , 2009, Math. Comput. Simul..

[17]  E. A. Sudicky,et al.  A simple iterative method for estimating evapotranspiration with integrated surface/subsurface flow models , 2015 .

[18]  James R. Craig,et al.  Assessing the performance of a semi-distributed hydrological model under various watershed discretization schemes , 2015 .

[19]  Alain Pietroniro,et al.  Grouped Response Units for Distributed Hydrologic Modeling , 1993 .

[20]  Willy Bauwens,et al.  Multi-variable sensitivity and identifiability analysis for a complex environmental model in view of integrated water quantity and water quality modeling. , 2012, Water science and technology : a journal of the International Association on Water Pollution Research.

[21]  P. Reed,et al.  Characterization of watershed model behavior across a hydroclimatic gradient , 2008 .

[22]  M. Webb,et al.  Quantification of modelling uncertainties in a large ensemble of climate change simulations , 2004, Nature.

[23]  Bryan A. Tolson,et al.  Assimilation of SMOS soil moisture over the Great Lakes basin , 2015 .

[24]  Noel A. C. Cressie,et al.  Statistics for Spatial Data: Cressie/Statistics , 1993 .

[25]  Max D. Morris,et al.  Factorial sampling plans for preliminary computational experiments , 1991 .

[26]  P. Krause,et al.  COMPARISON OF DIFFERENT EFFICIENCY CRITERIA FOR HYDROLOGICAL MODEL ASSESSMENT , 2005 .

[27]  Richard W. Healy,et al.  Factors influencing ground-water recharge in the eastern United States , 2007 .

[28]  Xiuying Wang,et al.  A framework for propagation of uncertainty contributed by parameterization, input data, model structure, and calibration/validation data in watershed modeling , 2014, Environ. Model. Softw..

[29]  William A. Brenneman,et al.  Optimal Sliced Latin Hypercube Designs , 2015, Technometrics.

[30]  Soroosh Sorooshian,et al.  Toward improved calibration of hydrologic models: Multiple and noncommensurable measures of information , 1998 .

[31]  Philip Marsh,et al.  Regionalisation of land surface hydrological model parameters in subarctic and arctic environments , 2008 .

[32]  Christine A. Shoemaker,et al.  Cannonsville Reservoir Watershed SWAT2000 model development, calibration and validation , 2007 .

[33]  W. James Shuttleworth,et al.  A fully multiple-criteria implementation of the Sobol' method for parameter sensitivity analysis , 2012 .

[34]  Amin Haghnegahdar,et al.  An Improved Framework for Watershed Discretization and Model Calibration: Application to the Great Lakes Basin , 2015 .

[35]  James R. Craig,et al.  Are all runoff processes the same? Numerical experiments comparing a Darcy‐Richards solver to an overland flow‐based approach for subsurface storm runoff simulation , 2015 .

[36]  D. Verseghy,et al.  The Canadian land surface scheme (CLASS): Its history and future , 2000 .

[37]  Bryan A. Tolson,et al.  An efficient framework for hydrologic model calibration on long data periods , 2013 .

[38]  Noel A Cressie,et al.  Statistics for Spatial Data, Revised Edition. , 1994 .

[39]  Saman Razavi,et al.  What do we mean by sensitivity analysis? The need for comprehensive characterization of “global” sensitivity in Earth and Environmental systems models , 2015 .

[40]  K. Abbaspour,et al.  Modelling blue and green water resources availability in Iran , 2009 .

[41]  A. Saltelli,et al.  Importance measures in global sensitivity analysis of nonlinear models , 1996 .