Distributed Evaluation of Local Sensitivity Analysis (DELSA), with application to hydrologic models

This paper presents a hybrid local‐global sensitivity analysis method termed the Distributed Evaluation of Local Sensitivity Analysis (DELSA), which is used here to identify important and unimportant parameters and evaluate how model parameter importance changes as parameter values change. DELSA uses derivative‐based “local” methods to obtain the distribution of parameter sensitivity across the parameter space, which promotes consideration of sensitivity analysis results in the context of simulated dynamics. This work presents DELSA, discusses how it relates to existing methods, and uses two hydrologic test cases to compare its performance with the popular global, variance‐based Sobol' method. The first test case is a simple nonlinear reservoir model with two parameters. The second test case involves five alternative “bucket‐style” hydrologic models with up to 14 parameters applied to a medium‐sized catchment (200 km2) in the Belgian Ardennes. Results show that in both examples, Sobol' and DELSA identify similar important and unimportant parameters, with DELSA enabling more detailed insight at much lower computational cost. For example, in the real‐world problem the time delay in runoff is the most important parameter in all models, but DELSA shows that for about 20% of parameter sets it is not important at all and alternative mechanisms and parameters dominate. Moreover, the time delay was identified as important in regions producing poor model fits, whereas other parameters were identified as more important in regions of the parameter space producing better model fits. The ability to understand how parameter importance varies through parameter space is critical to inform decisions about, for example, additional data collection and model development. The ability to perform such analyses with modest computational requirements provides exciting opportunities to evaluate complicated models as well as many alternative models.

[1]  John Doherty,et al.  Using Prediction Uncertainty Analysis to Design Hydrologic Monitoring Networks: Example Applications from the Great Lakes Water Availability Pilot Project , 2014 .

[2]  Claire R. Tiedeman,et al.  OPR-PPR, a Computer Program for Assessing Data Importance to Model Predictions Using Linear Statistics , 2014 .

[3]  Wei Gong,et al.  Assessing parameter importance of the Common Land Model based on qualitative and quantitative sensitivity analysis , 2013 .

[4]  Patrick M. Reed,et al.  Technical Note: Method of Morris effectively reduces the computational demands of global sensitivity analysis for distributed watershed models , 2013 .

[5]  Nilay Shah,et al.  Metamodelling with independent and dependent inputs , 2013, Comput. Phys. Commun..

[6]  Mauricio Zambrano-Bigiarini,et al.  A model-independent Particle Swarm Optimisation software for model calibration , 2013, Environ. Model. Softw..

[7]  Emanuele Borgonovo,et al.  Global sensitivity measures from given data , 2013, Eur. J. Oper. Res..

[8]  Juliane Mai,et al.  Use of eigendecomposition in a parameter sensitivity analysis of the Community Land Model , 2013 .

[9]  Jun Xia,et al.  An efficient integrated approach for global sensitivity analysis of hydrological model parameters , 2013, Environ. Model. Softw..

[10]  Patrick M. Reed,et al.  Time‐varying sensitivity analysis clarifies the effects of watershed model formulation on model behavior , 2013 .

[11]  Mary C. Hill,et al.  Evaluating model structure adequacy: The case of the Maggia Valley groundwater system, southern Switzerland , 2013 .

[12]  D. Benson,et al.  Particle tracking and the diffusion‐reaction equation , 2013 .

[13]  Carolina Massmann,et al.  Analysis of the behavior of a rainfall-runoff model using three global sensitivity analysis methods evaluated at different temporal scales , 2012 .

[14]  V. Guinot,et al.  Uncertainty analysis of river flooding and dam failure risks using local sensitivity computations , 2012, Reliab. Eng. Syst. Saf..

[15]  Oldrich Rakovec,et al.  State updating of a distributed hydrological model with Ensemble Kalman Filtering: Effects of updating frequency and observation network density on forecast accuracy , 2012 .

[16]  Ming Ye,et al.  Analysis of regression confidence intervals and Bayesian credible intervals for uncertainty quantification , 2012 .

[17]  Ming Ye,et al.  Towards a comprehensive assessment of model structural adequacy , 2012 .

[18]  W. James Shuttleworth,et al.  A fully multiple-criteria implementation of the Sobol' method for parameter sensitivity analysis , 2012 .

[19]  Paola Annoni,et al.  Estimation of global sensitivity indices for models with dependent variables , 2012, Comput. Phys. Commun..

[20]  Willy Bauwens,et al.  Sobol' sensitivity analysis of a complex environmental model , 2011, Environ. Model. Softw..

[21]  G. Gertner,et al.  Reliability of global sensitivity indices , 2011 .

[22]  Dmitri Kavetski,et al.  Hydrological field data from a modeller's perspective: Part 2: process‐based evaluation of model hypotheses , 2011 .

[23]  Hidde Leijnse,et al.  Radar rainfall estimation of stratiform winter precipitation in the Belgian Ardennes , 2011 .

[24]  Andrej Pázman,et al.  Nonlinear Regression , 2019, Handbook of Regression Analysis With Applications in R.

[25]  Dmitri Kavetski,et al.  Ancient numerical daemons of conceptual hydrological modeling: 1. Fidelity and efficiency of time stepping schemes , 2010 .

[26]  M. Clark,et al.  Ancient numerical daemons of conceptual hydrological modeling: 2. Impact of time stepping schemes on model analysis and prediction , 2010 .

[27]  Ming Ye,et al.  A Model‐Averaging Method for Assessing Groundwater Conceptual Model Uncertainty , 2010, Ground water.

[28]  I. Sobol,et al.  A new derivative based importance criterion for groups of variables and its link with the global sensitivity indices , 2010, Comput. Phys. Commun..

[29]  John Doherty,et al.  A short exploration of structural noise , 2010 .

[30]  Remko Uijlenhoet,et al.  The hydrological response of the Ourthe catchment to climate change as modelled by the HBV model , 2009 .

[31]  Constantinos C. Pantelides,et al.  Monte Carlo evaluation of derivative-based global sensitivity measures , 2009, Reliab. Eng. Syst. Saf..

[32]  Mary C. Hill,et al.  Sensitivity analysis, calibration, and testing of a distributed hydrological model using error‐based weighting and one objective function , 2009 .

[33]  Sergei S. Kucherenko,et al.  Derivative based global sensitivity measures and their link with global sensitivity indices , 2009, Math. Comput. Simul..

[34]  Martyn P. Clark,et al.  Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models , 2008 .

[35]  Florian Pappenberger,et al.  Multi‐method global sensitivity analysis (MMGSA) for modelling floodplain hydrological processes , 2008 .

[36]  Ning Liu,et al.  Inverse Theory for Petroleum Reservoir Characterization and History Matching , 2008 .

[37]  Saltelli Andrea,et al.  Global Sensitivity Analysis: The Primer , 2008 .

[38]  P. Reed,et al.  Characterization of watershed model behavior across a hydroclimatic gradient , 2008 .

[39]  Florian Pappenberger,et al.  Multi-method global sensitivity analysis of flood inundation models. , 2008 .

[40]  Charles B. Andrews,et al.  Effective Groundwater Model Calibration: With Analysis of Data, Sensitivities, Predictions, and Uncertainty , 2007 .

[41]  L Foglia,et al.  Testing Alternative Ground Water Models Using Cross‐Validation and Other Methods , 2007, Ground water.

[42]  Emanuele Borgonovo,et al.  A new uncertainty importance measure , 2007, Reliab. Eng. Syst. Saf..

[43]  George Kuczera,et al.  Model smoothing strategies to remove microscale discontinuities and spurious secondary optima in objective functions in hydrological calibration , 2007 .

[44]  C. Tiedeman,et al.  Effective Groundwater Model Calibration , 2007 .

[45]  M. Kenward,et al.  An Introduction to the Bootstrap , 2007 .

[46]  T. Vesala,et al.  Towards a standardized processing of Net Ecosystem Exchange measured with eddy covariance technique: algorithms and uncertainty estimation , 2006 .

[47]  P. Reed,et al.  Hydrology and Earth System Sciences Discussions Comparing Sensitivity Analysis Methods to Advance Lumped Watershed Model Identification and Evaluation , 2022 .

[48]  Keith Beven,et al.  Influence of uncertain boundary conditions and model structure on flood inundation predictions. , 2006 .

[49]  E. Borgonovo Measuring Uncertainty Importance: Investigation and Comparison of Alternative Approaches , 2006, Risk analysis : an official publication of the Society for Risk Analysis.

[50]  N. A. S. Hamm,et al.  Variance-based sensitivity analysis of the probability of hydrologically induced slope instability , 2006, Comput. Geosci..

[51]  R. Srinivasan,et al.  A global sensitivity analysis tool for the parameters of multi-variable catchment models , 2006 .

[52]  S. Grunwald,et al.  A global sensitivity analysis tool for the parameters of multivariable catchment models , 2006 .

[53]  T. Vesala,et al.  On the separation of net ecosystem exchange into assimilation and ecosystem respiration: review and improved algorithm , 2005 .

[54]  G. Seber,et al.  Nonlinear Regression: Seber/Nonlinear Regression , 2005 .

[55]  Clifford H. Thurber,et al.  Parameter estimation and inverse problems , 2005 .

[56]  Albert Tarantola,et al.  Inverse problem theory - and methods for model parameter estimation , 2004 .

[57]  D. M. Ely,et al.  A method for evaluating the importance of system state observations to model predictions, with application to the Death Valley regional groundwater flow system , 2004 .

[58]  A. O'Hagan,et al.  Probabilistic sensitivity analysis of complex models: a Bayesian approach , 2004 .

[59]  Frances Y. Kuo,et al.  Remark on algorithm 659: Implementing Sobol's quasirandom sequence generator , 2003, TOMS.

[60]  Neil McIntyre,et al.  Towards reduced uncertainty in conceptual rainfall‐runoff modelling: dynamic identifiability analysis , 2003 .

[61]  C. Tiedeman,et al.  Methods for using groundwater model predictions to guide hydrogeologic data collection, with application to the Death Valley regional groundwater flow system , 2003 .

[62]  A. Saltelli,et al.  Making best use of model evaluations to compute sensitivity indices , 2002 .

[63]  Harald Kunstmann,et al.  Conditional first‐order second‐moment method and its application to the quantification of uncertainty in groundwater modeling , 2002 .

[64]  Stefano Tarantola,et al.  Sensitivity Analysis in Practice , 2002 .

[65]  M. Aubinet,et al.  Long term carbon dioxide exchange above a mixed forest in the Belgian Ardennes , 2001 .

[66]  Willem Bouten,et al.  Information content of time domain reflectometry waveforms , 2001 .

[67]  M. Hill,et al.  A Comparison of Solute‐Transport Solution Techniques and Their Effect on Sensitivity Analysis and Inverse Modeling Results , 2001, Ground water.

[68]  I. Sobola,et al.  Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates , 2001 .

[69]  I. Sobol Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates , 2001 .

[70]  Willem Bouten,et al.  Information content of data for identifying soil hydraulic parameters from outflow experiments , 2001 .

[71]  Moon-Hyun Chun,et al.  An uncertainty importance measure using a distance metric for the change in a cumulative distribution function , 2000, Reliab. Eng. Syst. Saf..

[72]  Willem Bouten,et al.  A method for identifying optimum strategies of measuring soil water contents for calibrating a root water uptake model , 2000 .

[73]  R. Katz Extreme value theory for precipitation: sensitivity analysis for climate change , 1999 .

[74]  J. C. Helton,et al.  Statistical Analyses of Scatterplots to Identify Important Factors in Large-Scale Simulations, 1: Review and Comparison of Techniques , 1999 .

[75]  Stefano Tarantola,et al.  A Quantitative Model-Independent Method for Global Sensitivity Analysis of Model Output , 1999, Technometrics.

[76]  N. Draper,et al.  Applied Regression Analysis: Draper/Applied Regression Analysis , 1998 .

[77]  I. Sobol,et al.  Sensitivity Measures, ANOVA-like Techniques and the Use of Bootstrap , 1997 .

[78]  Mary C. Hill,et al.  Death valley regional ground-water flow model calibration using optimal parameter estimation methods and geoscientific information systems , 1999 .

[79]  Mary C. Hill,et al.  Two-dimensional advective transport in ground-water flow parameter estimation , 1996 .

[80]  K. Beven,et al.  Bayesian Estimation of Uncertainty in Runoff Prediction and the Value of Data: An Application of the GLUE Approach , 1996 .

[81]  A. Saltelli,et al.  Importance measures in global sensitivity analysis of nonlinear models , 1996 .

[82]  Harvey M. Wagner,et al.  Global Sensitivity Analysis , 1995, Oper. Res..

[83]  Kwang-Il Ahn,et al.  A new approach for measuring uncertainty importance and distributional sensitivity in probabilistic safety assessment , 1994 .

[84]  Jon C. Helton,et al.  Uncertainty and sensitivity analysis techniques for use in performance assessment for radioactive waste disposal , 1993 .

[85]  George E. P. Box,et al.  Bayesian Inference in Statistical Analysis: Box/Bayesian , 1992 .

[86]  Max D. Morris,et al.  Factorial sampling plans for preliminary computational experiments , 1991 .

[87]  Paul Bratley,et al.  Algorithm 659: Implementing Sobol's quasirandom sequence generator , 1988, TOMS.

[88]  W. Menke Geophysical data analysis , 1984 .

[89]  W. Menke Geophysical data analysis : discrete inverse theory , 1984 .

[90]  S. Weisberg,et al.  Residuals and Influence in Regression , 1982 .

[91]  G. Hornberger,et al.  Approach to the preliminary analysis of environmental systems , 1981 .

[92]  G. Hornberger,et al.  Empirical equations for some soil hydraulic properties , 1978 .

[93]  K.,et al.  Nonlinear sensitivity analysis of multiparameter model systems , 1977 .

[94]  K. Shuler,et al.  Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. III. Analysis of the approximations , 1975 .

[95]  Franklin A. Graybill,et al.  Introduction to the Theory of Statistics, 3rd ed. , 1974 .

[96]  C. Fortuin,et al.  Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. I Theory , 1973 .

[97]  G. C. Tiao,et al.  Bayesian inference in statistical analysis , 1973 .

[98]  D. M. Ellis,et al.  Applied Regression Analysis , 1968 .

[99]  J. Wolfowitz,et al.  An Introduction to the Theory of Statistics , 1951, Nature.