Revisiting the Basis of Sensitivity Analysis for Dynamical Earth System Models

This paper investigates the problem of global sensitivity analysis (GSA) of Dynamical Earth System Models and proposes a basis for how such analyses should be performed. We argue that (a) performance metric‐based approaches to parameter GSA are actually identifiability analyses, (b) the use of a performance metric to assess sensitivity unavoidably distorts the information provided by the model about relative parameter importance, and (c) it is a serious conceptual flaw to interpret the results of such an analysis as being consistent and accurate indications of the sensitivity of the model response to parameter perturbations. Further, because such approaches depend on availability of system state/output observational data, the analysis they provide is necessarily incomplete. Here we frame the GSA problem from first principles, using trajectories of the partial derivatives of model outputs with respect to controlling factors as the theoretical basis for sensitivity, and construct a global sensitivity matrix from which statistical indices of total period time‐aggregate parameter importance, and time series of time‐varying parameter importance, can be inferred. We demonstrate this framework using the HBV‐SASK conceptual hydrologic model applied to the Oldman basin in Canada and show that it disagrees with performance metric‐based methods regarding which parameters exert the strongest controls on model behavior. Further, it is highly efficient, requiring less than 1,000 base samples to obtain stable and robust parameter importance assessments for our 10‐parameter example.

[1]  Saman Razavi,et al.  VARS-TOOL: A toolbox for comprehensive, efficient, and robust sensitivity and uncertainty analysis , 2019, Environ. Model. Softw..

[2]  Saman Razavi,et al.  Global sensitivity analysis for high-dimensional problems: How to objectively group factors and measure robustness and convergence while reducing computational cost , 2019, Environ. Model. Softw..

[3]  Alberto Guadagnini,et al.  Moment-based metrics for global sensitivity analysis of hydrological systems , 2017 .

[4]  Saman Razavi,et al.  Multicriteria sensitivity analysis as a diagnostic tool for understanding model behaviour and characterizing model uncertainty , 2017 .

[5]  Saman Razavi,et al.  Insights into sensitivity analysis of Earth and environmental systems models: On the impact of parameter perturbation scale , 2017, Environ. Model. Softw..

[6]  Emanuele Borgonovo,et al.  Making the most out of a hydrological model data set: Sensitivity analyses to open the model black‐box , 2017 .

[7]  Saman Razavi,et al.  Progressive Latin Hypercube Sampling: An efficient approach for robust sampling-based analysis of environmental models , 2017, Environ. Model. Softw..

[8]  P. Bates,et al.  Quantifying the importance of spatial resolution and other factors through global sensitivity analysis of a flood inundation model , 2016 .

[9]  Martin Volk,et al.  On characterizing the temporal dominance patterns of model parameters and processes , 2016 .

[10]  Philippe Gourbesville,et al.  Spatial Global Sensitivity Analysis of High Resolution classified topographic data use in 2D urban flood modelling , 2016, Environ. Model. Softw..

[11]  H. Gupta,et al.  A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory , 2016 .

[12]  H. Gupta,et al.  A new framework for comprehensive, robust, and efficient global sensitivity analysis: 2. Application , 2016 .

[13]  Jim W. Hall,et al.  Sensitivity analysis of environmental models: A systematic review with practical workflow , 2014, Environ. Model. Softw..

[14]  Francesca Pianosi,et al.  Understanding the time‐varying importance of different uncertainty sources in hydrological modelling using global sensitivity analysis , 2016 .

[15]  Jeremy Rohmer,et al.  Evaluating uncertainties of future marine flooding occurrence as sea-level rises , 2015, Environ. Model. Softw..

[16]  Saman Razavi,et al.  What do we mean by sensitivity analysis? The need for comprehensive characterization of “global” sensitivity in Earth and Environmental systems models , 2015 .

[17]  Francesca Pianosi,et al.  A simple and efficient method for global sensitivity analysis based on cumulative distribution functions , 2015, Environ. Model. Softw..

[18]  Hoshin Vijai Gupta,et al.  Debates—the future of hydrological sciences: A (common) path forward? Using models and data to learn: A systems theoretic perspective on the future of hydrological science , 2014 .

[19]  N. Fohrer,et al.  How to improve the representation of hydrological processes in SWAT for a lowland catchment – temporal analysis of parameter sensitivity and model performance , 2014 .

[20]  Mary C. Hill,et al.  Distributed Evaluation of Local Sensitivity Analysis (DELSA), with application to hydrologic models , 2014 .

[21]  Thorsten Wagener,et al.  A new approach to visualizing time-varying sensitivity indices for environmental model diagnostics across evaluation time-scales , 2014, Environ. Model. Softw..

[22]  Patrick M. Reed,et al.  From maps to movies: High-resolution time-varying sensitivity analysis for spatially distributed watershed models , 2013 .

[23]  B. Croke,et al.  Addressing ten questions about conceptual rainfall–runoff models with global sensitivity analyses in R , 2013 .

[24]  Hoshin Vijai Gupta,et al.  Large-sample hydrology: a need to balance depth with breadth , 2013 .

[25]  Patrick M. Reed,et al.  Time‐varying sensitivity analysis clarifies the effects of watershed model formulation on model behavior , 2013 .

[26]  W. James Shuttleworth,et al.  A fully multiple-criteria implementation of the Sobol' method for parameter sensitivity analysis , 2012 .

[27]  Dmitri Kavetski,et al.  Pursuing the method of multiple working hypotheses for hydrological modeling , 2011 .

[28]  Erwin Zehe,et al.  Inferring model structural deficits by analyzing temporal dynamics of model performance and parameter sensitivity , 2011 .

[29]  Erwin Zehe,et al.  Temporal dynamics of model parameter sensitivity for computationally expensive models with the Fourier amplitude sensitivity test , 2011 .

[30]  David Makowski,et al.  Multivariate sensitivity analysis to measure global contribution of input factors in dynamic models , 2011, Reliab. Eng. Syst. Saf..

[31]  Indrajeet Chaubey,et al.  Sensitivity and identifiability of stream flow generation parameters of the SWAT model , 2010 .

[32]  Bertrand Iooss,et al.  Global sensitivity analysis for models with spatially dependent outputs , 2009, 0911.1189.

[33]  Hoshin Vijai Gupta,et al.  Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling , 2009 .

[34]  Matieyendou Lamboni,et al.  Multivariate global sensitivity analysis for dynamic crop models , 2009 .

[35]  Sergei S. Kucherenko,et al.  Derivative based global sensitivity measures and their link with global sensitivity indices , 2009, Math. Comput. Simul..

[36]  Hoshin Vijai Gupta,et al.  A process‐based diagnostic approach to model evaluation: Application to the NWS distributed hydrologic model , 2008 .

[37]  Yuqiong Liu,et al.  Reconciling theory with observations: elements of a diagnostic approach to model evaluation , 2008 .

[38]  P. Reed,et al.  Rainfall characteristics define the value of streamflow observations for distributed watershed model identification , 2008 .

[39]  Florian Pappenberger,et al.  Multi‐method global sensitivity analysis (MMGSA) for modelling floodplain hydrological processes , 2008 .

[40]  Saltelli Andrea,et al.  Global Sensitivity Analysis: The Primer , 2008 .

[41]  P. Reed,et al.  Characterization of watershed model behavior across a hydroclimatic gradient , 2008 .

[42]  Florian Pappenberger,et al.  Multi-method global sensitivity analysis of flood inundation models. , 2008 .

[43]  Andrea Saltelli,et al.  An effective screening design for sensitivity analysis of large models , 2007, Environ. Model. Softw..

[44]  Bart Nijssen,et al.  Monte Carlo sensitivity analysis of land surface parameters using the Variable Infiltration Capacity model , 2007 .

[45]  Brian J. Williams,et al.  Sensitivity analysis when model outputs are functions , 2006, Reliab. Eng. Syst. Saf..

[46]  R. Srinivasan,et al.  A global sensitivity analysis tool for the parameters of multi-variable catchment models , 2006 .

[47]  John S. Selker,et al.  Information, artifacts, and noise in dQ/dt − Q recession analysis , 2006 .

[48]  S. Uhlenbrook,et al.  Sensitivity analyses of a distributed catchment model to verify the model structure , 2005 .

[49]  A. Sankarasubramanian,et al.  Validation of a watershed model without calibration , 2003 .

[50]  Murugesu Sivapalan,et al.  Downward approach to hydrological prediction , 2003 .

[51]  S. Sorooshian,et al.  Effective and efficient algorithm for multiobjective optimization of hydrologic models , 2003 .

[52]  Neil McIntyre,et al.  Towards reduced uncertainty in conceptual rainfall‐runoff modelling: dynamic identifiability analysis , 2003 .

[53]  Soroosh Sorooshian,et al.  Toward improved calibration of hydrologic models: Combining the strengths of manual and automatic methods , 2000 .

[54]  I. Sobol,et al.  On the use of variance reducing multipliers in Monte Carlo computations of a global sensitivity index , 1999 .

[55]  Göran Lindström,et al.  Development and test of the distributed HBV-96 hydrological model , 1997 .

[56]  Allen F. Moench,et al.  Specific Yield as Determined by Type‐Curve Analysis of Aquifer‐Test Data , 1994 .

[57]  Max D. Morris,et al.  Factorial sampling plans for preliminary computational experiments , 1991 .

[58]  S. Sorooshian,et al.  The Automatic Calibration of Conceptual Catchment Models Using Derivative‐Based Optimization Algorithms , 1985 .

[59]  K.,et al.  Nonlinear sensitivity analysis of multiparameter model systems , 1977 .

[60]  Yonathan Bard,et al.  Nonlinear parameter estimation , 1974 .

[61]  J. Nash,et al.  River flow forecasting through conceptual models part I — A discussion of principles☆ , 1970 .

[62]  D. Cox,et al.  An Analysis of Transformations , 1964 .