Model emulation and moment-independent sensitivity analysis: An application to environmental modelling

Moment-independent sensitivity methods are attracting increasing attention among practitioners, since they provide a thorough way of investigating the sensitivity of model output under uncertainty. However, their estimation is challenging, especially in the presence of computationally intensive models. We argue that replacement of the original model by a metamodel can contribute in lowering the computation burden. A numerical estimation procedure is set forth. The procedure is first tested on analytical cases with increased structural complexity. We utilize the emulator proposed in Ratto and Pagano (2010). Results show that the emulator allows an accurate estimation of density-based sensitivity measures, when the main structural features of the original model are captured. However, performance deteriorates for a model with interactions of order higher than 2. For this test case, also a kriging emulator is investigated, but no gain in performance is registered. However, an accurate estimation is obtained by applying a logarithmic transformation of the model output for both the kriging and Ratto and Pagano (2010) emulators. These findings are then applied to the investigation of a benchmark environmental case study, the LevelE model. Results show that use of the metamodel allows an efficient estimation of moment-independent sensitivity measures while leading to a notable reduction in computational burden.

[1]  A. Saltelli,et al.  Non-parametric statistics in sensitivity analysis for model output: A comparison of selected techniques , 1990 .

[2]  Dapeng Yu.  Parallelization of a two-dimensional flood inundation model based on domain decomposition , 2010, Environ. Model. Softw..

[3]  Thomas J. Santner,et al.  The Design and Analysis of Computer Experiments , 2003, Springer Series in Statistics.

[4]  Bruno Sudret,et al.  Global sensitivity analysis using polynomial chaos expansions , 2008, Reliab. Eng. Syst. Saf..

[5]  N. Glick.  Measurements of separation among probability densities or random variables , 1975 .

[6]  R. Tibshirani,et al.  Generalized Additive Models , 1986 .

[7]  N. L. Johnson,et al.  Continuous Multivariate Distributions, Volume 1: Models and Applications , 2019 .

[8]  Moon-Hyun Chun,et al.  An uncertainty importance measure using a distance metric for the change in a cumulative distribution function , 2000, Reliab. Eng. Syst. Saf..

[9]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[10]  Jerome H. Friedman.  Multivariate adaptive regression splines (with discussion) , 1991 .

[11]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[12]  Jon C. Helton,et al.  Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models , 2009, Reliab. Eng. Syst. Saf..

[13]  Saltelli Andrea,et al.  Global Sensitivity Analysis: The Primer , 2008 .

[14]  Florian Pappenberger,et al.  Multi-method global sensitivity analysis of flood inundation models. , 2008 .

[15]  Andreas Griewank,et al.  Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition , 2000, Frontiers in applied mathematics.

[16]  Emanuele Borgonovo,et al.  A new uncertainty importance measure , 2007, Reliab. Eng. Syst. Saf..

[17]  M. Friedman.  Greedy Fun tion Approximation : A Gradient Boosting , 1999 .

[18]  Anthony J. Jakeman,et al.  Sensitivity analysis for assessing the behaviour of a landscape-based sediment source and transport model , 2003, Environ. Model. Softw..

[19]  M. Ratto,et al.  Using recursive algorithms for the efficient identification of smoothing spline ANOVA models , 2010 .

[20]  I. Sobol,et al.  About the use of rank transformation in sensitivity analysis of model output , 1995 .

[21]  F. L. Dimet,et al.  Sensitivity analysis and parameter estimation for distributed hydrological modeling: potential of variational methods , 2009 .

[22]  J. Simonoff.  Smoothing Methods in Statistics , 1998 .

[23]  Terry Andres,et al.  Sensitivity analysis of model output: an investigation of new techniques , 1993 .

[24]  Gemma Manache,et al.  Identification of reliable regression- and correlation-based sensitivity measures for importance ranking of water-quality model parameters , 2008, Environ. Model. Softw..

[25]  D. Krige.  A statistical approach to some basic mine valuation problems on the Witwatersrand, by D.G. Krige, published in the Journal, December 1951 : introduction by the author , 1951 .

[26]  Qiao Liu,et al.  A new computational method of a moment-independent uncertainty importance measure , 2009, Reliab. Eng. Syst. Saf..

[27]  John P. Norton.  Algebraic sensitivity analysis of environmental models , 2008, Environ. Model. Softw..

[28]  H Rabitz,et al.  Systems Analysis at the Molecular Scale , 1989, Science.

[29]  Peter C. Young,et al.  State Dependent Parameter metamodelling and sensitivity analysis , 2007, Comput. Phys. Commun..

[30]  Robert E. Shannon,et al.  Design and analysis of simulation experiments , 1978, WSC '78.

[31]  Adrian Sandu,et al.  Application of ADIFOR for air pollution model sensitivity studies , 2000, Environ. Model. Softw..

[32]  Emanuele Borgonovo,et al.  Sampling plans for the estimation of moment-independent importance measures , 2010 .

[33]  Philip Rabinowitz,et al.  Methods of Numerical Integration , 1985 .

[34]  Bernard W. Silverman,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[35]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[36]  A. Saltelli,et al.  Importance measures in global sensitivity analysis of nonlinear models , 1996 .

[37]  Kwang-Il Ahn,et al.  A new approach for measuring uncertainty importance and distributional sensitivity in probabilistic safety assessment , 1994 .

[38]  Chong Gu.  Smoothing Spline Anova Models , 2002 .

[39]  Paola Annoni,et al.  Sixth International Conference on Sensitivity Analysis of Model Output How to avoid a perfunctory sensitivity analysis , 2010 .

[40]  Jon C. Helton,et al.  Uncertainty and sensitivity analysis techniques for use in performance assessment for radioactive waste disposal , 1993 .

[41]  S. Hora,et al.  A Robust Measure of Uncertainty Importance for Use in Fault Tree System Analysis , 1990 .

[42]  Peter C. Young,et al.  Non-parametric estimation of conditional moments for sensitivity analysis , 2009, Reliab. Eng. Syst. Saf..

[43]  Samuel Buis,et al.  Global sensitivity analysis measures the quality of parameter estimation: The case of soil parameters and a crop model , 2010, Environ. Model. Softw..

[44]  Kai W. Wirtz,et al.  Linear understanding of a huge aquatic ecosystem model using a group-collecting sensitivity analysis , 2002, Environ. Model. Softw..

[45]  Brian J Reich,et al.  Surface Estimation, Variable Selection, and the Nonparametric Oracle Property. , 2011, Statistica Sinica.

[46]  Sonja Kuhnt,et al.  Design and analysis of computer experiments , 2010 .

[47]  J. Friedman.  Multivariate adaptive regression splines , 1990 .

[48]  S. Tarantola,et al.  Moment Independent Importance Measures: New Results and Analytical Test Cases , 2011, Risk analysis : an official publication of the Society for Risk Analysis.

[49]  Anna Nagurney,et al.  Foundations of Financial Economics , 1997 .

[50]  V. Estrada,et al.  Global sensitivity analysis in the development of first principle-based eutrophication models , 2010, Environ. Model. Softw..

[51]  Ilya M. Sobol,et al.  Sensitivity Estimates for Nonlinear Mathematical Models , 1993 .

[52]  Jon C. Helton,et al.  Multiple predictor smoothing methods for sensitivity analysis: Description of techniques , 2008, Reliab. Eng. Syst. Saf..

[53]  Roger D. Braddock,et al.  Sensitivity analysis of the IMAGE Greenhouse model , 1999, Environ. Model. Softw..

[54]  Søren Nymand Lophaven,et al.  DACE - A Matlab Kriging Toolbox, Version 2.0 , 2002 .

[55]  Andrea Saltelli,et al.  An effective screening design for sensitivity analysis of large models , 2007, Environ. Model. Softw..

[56]  D. L. Wallace.  Asymptotic Approximations to Distributions , 1958 .

[57]  Jean-François Richard,et al.  Methods of Numerical Integration , 2000 .

[58]  H. Rabitz,et al.  General foundations of high‐dimensional model representations , 1999 .

[59]  R. Strawderman.  Continuous Multivariate Distributions, Volume 1: Models and Applications , 2001 .

[60]  Bruno Sudret,et al.  Efficient computation of global sensitivity indices using sparse polynomial chaos expansions , 2010, Reliab. Eng. Syst. Saf..

[61]  Jack P. C. Kleijnen,et al.  Kriging Metamodeling in Simulation: A Review , 2007, Eur. J. Oper. Res..

[62]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[63]  R. Tibshirani,et al.  Generalized Additive Models , 1991 .

[64]  H. Rabitz,et al.  High Dimensional Model Representations Generated from Low Dimensional Data Samples. I. mp-Cut-HDMR , 2001 .

[65]  S. A. Cryer,et al.  Regional sensitivity analysis using a fractional factorial method for the USDA model GLEAMS , 1999, Environ. Model. Softw..

[66]  H. Jeffreys,et al.  The Law of Error , 1938, Nature.

[67]  A. Saltelli,et al.  On the Relative Importance of Input Factors in Mathematical Models , 2002 .

[68]  Andrea Saltelli,et al.  Sensitivity analysis didn't help. A practitioner's critique of the Stern review , 2010 .

[69]  Jim Berger,et al.  Special Issue on Computer Modeling , 2009, Technometrics.

[70]  YuDapeng.  Parallelization of a two-dimensional flood inundation model based on domain decomposition , 2010 .

[71]  E. Borgonovo.  Measuring Uncertainty Importance: Investigation and Comparison of Alternative Approaches , 2006, Risk analysis : an official publication of the Society for Risk Analysis.

[72]  Alison S. Tomlin,et al.  GUI-HDMR - A software tool for global sensitivity analysis of complex models , 2009, Environ. Model. Softw..

[73]  Peter C. Young,et al.  The Identification and Estimation of Nonlinear Stochastic Systems , 2001 .

[74]  Stefano Tarantola,et al.  Sensitivity analysis of the rice model WARM in Europe: Exploring the effects of different locations, climates and methods of analysis on model sensitivity to crop parameters , 2010, Environ. Model. Softw..

[75]  D. Cacuci.  Sensitivity theory for nonlinear systems. I. Nonlinear functional analysis approach , 1981 .

[76]  S. Tarantola,et al.  Moment independent and variance‐based sensitivity analysis with correlations: An application to the stability of a chemical reactor , 2008 .

[77]  Silvio Funtowicz,et al.  Application of a checklist for quality assistance in environmental modelling to an energy model , 2005 .

[78]  A. O'Hagan,et al.  Probabilistic sensitivity analysis of complex models: a Bayesian approach , 2004 .

[79]  Emanuele Borgonovo,et al.  A Study of Interactions in the Risk Assessment of Complex Engineering Systems: An Application to Space PSA , 2011, Oper. Res..