Simple approach to emulating complex computer models for global sensitivity analysis

Sensitivity analysis is an important step in understanding how uncertainty is propagated through complex computer models. Unfortunately, the most reliable sensitivity analysis techniques take a significant amount of time to execute due to the large number of computer model evaluations required. Emulators can be used to speed up the process by replacing the computer model with a statistical model that mimics the computer model and is computationally efficient. In this manuscript we propose two emulator-based sensitivity index estimators that require minimal set-up and are computationally inexpensive to compute. We demonstrate their accuracy with computer models that have known sensitivity index values and illustrate their application in practice with the agriculture systems simulator APSIM. We propose two methods for estimating sensitivity indices based on a simple emulator.Our methods can easily be implemented in R and do not require prior specification.They are more accurate than existing Monte Carlo methods with the same number of model runs.We illustrate our methods using the agriculture systems simulator APSIM.

[1]  Stefano Tarantola,et al.  Estimating the approximation error when fixing unessential factors in global sensitivity analysis , 2007, Reliab. Eng. Syst. Saf..

[2]  C. Fortuin,et al.  Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. I Theory , 1973 .

[3]  Paola Annoni,et al.  Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index , 2010, Comput. Phys. Commun..

[4]  Bruno Sudret,et al.  Adaptive sparse polynomial chaos expansion based on least angle regression , 2011, J. Comput. Phys..

[5]  Jon C. Helton,et al.  Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models , 2009, Reliab. Eng. Syst. Saf..

[6]  Peter C. Young,et al.  Non-parametric estimation of conditional moments for sensitivity analysis , 2009, Reliab. Eng. Syst. Saf..

[7]  R. Tibshirani,et al.  Generalized Additive Models , 1991 .

[8]  Brian J. Williams,et al.  Sensitivity analysis when model outputs are functions , 2006, Reliab. Eng. Syst. Saf..

[9]  Thierry Alex Mara,et al.  Comparison of some efficient methods to evaluate the main effect of computer model factors , 2008 .

[10]  Bruno Sudret,et al.  Global sensitivity analysis using polynomial chaos expansions , 2008, Reliab. Eng. Syst. Saf..

[11]  Peter C. Young,et al.  State Dependent Parameter metamodelling and sensitivity analysis , 2007, Comput. Phys. Commun..

[12]  Anthony O'Hagan,et al.  Diagnostics for Gaussian Process Emulators , 2009, Technometrics.

[13]  Gregery T. Buzzard.  Efficient Basis Change for Sparse-Grid Interpolating Polynomials with Application to T-Cell Sensitivity Analysis , 2013 .

[14]  S. Wood.  Generalized Additive Models: An Introduction with R , 2006 .

[15]  Michael S. Eldred,et al.  DAKOTA : a multilevel parallel object-oriented framework for design optimization, parameter estimation, uncertainty quantification, and sensitivity analysis. Version 5.0, user's reference manual. , 2010 .

[16]  Senthold Asseng,et al.  An overview of APSIM, a model designed for farming systems simulation , 2003 .

[17]  Chris Murphy,et al.  APSIM - Evolution towards a new generation of agricultural systems simulation , 2014, Environ. Model. Softw..

[18]  B. Bryan,et al.  Sensitivity and uncertainty analysis of the APSIM-wheat model: interactions between cultivar, environmental, and management parameters. , 2014 .

[19]  A. O'Hagan,et al.  Probabilistic sensitivity analysis of complex models: a Bayesian approach , 2004 .

[20]  Peter Carberry,et al.  Crop sequences in Australia’s northern grain zone are less agronomically efficient than implied by the sum of their parts , 2014 .

[21]  M. Ratto,et al.  Using recursive algorithms for the efficient identification of smoothing spline ANOVA models , 2010 .

[22]  John O. Carter,et al.  Using spatial interpolation to construct a comprehensive archive of Australian climate data , 2001, Environ. Model. Softw..

[23]  Herschel Rabitz,et al.  General formulation of HDMR component functions with independent and correlated variables , 2011, Journal of Mathematical Chemistry.

[24]  M. Jansen.  Analysis of variance designs for model output , 1999 .

[25]  Dongbin Xiu,et al.  Stochastic Collocation Methods on Unstructured Grids in High Dimensions via Interpolation , 2012, SIAM J. Sci. Comput..

[26]  T. Ishigami,et al.  An importance quantification technique in uncertainty analysis for computer models , 1990, [1990] Proceedings. First International Symposium on Uncertainty Modeling and Analysis.

[27]  Max D. Morris,et al.  Factorial sampling plans for preliminary computational experiments , 1991 .

[28]  Herschel Rabitz,et al.  Sixth International Conference on Sensitivity Analysis of Model Output Global Sensitivity Analysis for Systems with Independent and / or Correlated Inputs , 2013 .

[29]  Jon C. Helton,et al.  Multiple predictor smoothing methods for sensitivity analysis: Description of techniques , 2008, Reliab. Eng. Syst. Saf..

[30]  P. Thorburn,et al.  Using the APSIM model to estimate nitrous oxide emissions from diverse Australian sugarcane production systems. , 2010 .

[31]  Jeremy E. Oakley,et al.  Estimating Multiparameter Partial Expected Value of Perfect Information from a Probabilistic Sensitivity Analysis Sample , 2013, Medical decision making : an international journal of the Society for Medical Decision Making.

[32]  N. Huth,et al.  Impacts of fertilisers and legumes on N2O and CO2 emissions from soils in subtropical agricultural systems: A simulation study , 2010 .