Transformations and invariance in the sensitivity analysis of computer experiments

Monotonic transformations are widely employed in statistics and data analysis. In computer experiments they are often used to gain accuracy in the estimation of global sensitivity statistics. However, one faces the question of interpreting results that are obtained on the transformed data back on the original data. The situation is even more complex in computer experiments, because transformations alter the model input–output mapping and distort the estimators. This work demonstrates that the problem can be solved by utilizing statistics which are monotonic transformation invariant. To do so, we offer an investigation into the families of metrics either based on densities or on cumulative distribution functions that are monotonic transformation invariant and we introduce a new generalized family of metrics. Numerical experiments show that transformations allow numerical convergence in the estimates of global sensitivity statistics, both invariant and not, in cases in which it would otherwise be impossible to obtain convergence. However, one fully exploits the increased numerical accuracy if the global sensitivity statistic is monotonic transformation invariant. Conversely, estimators of measures that do not have this invariance property might lead to misleading deductions.

[1]  Ola Mahmoud,et al.  A Probability Metrics Approach to Financial Risk Measures , 2016 .

[2]  J. Oakley,et al.  An Efficient Method for Computing Single-Parameter Partial Expected Value of Perfect Information , 2013, Medical decision making : an international journal of the Society for Medical Decision Making.

[3]  Emanuele Borgonovo,et al.  Invariant Probabilistic Sensitivity Analysis , 2013, Manag. Sci..

[4]  Emanuele Borgonovo,et al.  Global sensitivity measures from given data , 2013, Eur. J. Oper. Res..

[5]  Emanuele Borgonovo,et al.  Sampling strategies in density-based sensitivity analysis , 2012, Environ. Model. Softw..

[6]  Elmar Plischke,et al.  How to compute variance-based sensitivity indicators with your spreadsheet software , 2012, Environ. Model. Softw..

[7]  Andrea Castelletti,et al.  Emulation techniques for the reduction and sensitivity analysis of complex environmental models , 2012, Environ. Model. Softw..

[8]  A. Owen.  Variance Components and Generalized Sobol' Indices , 2012, SIAM/ASA J. Uncertain. Quantification.

[9]  A. Saltelli,et al.  Update 1 of: Sensitivity analysis for chemical models. , 2012, Chemical reviews.

[10]  Art B. Owen,et al.  Better estimation of small sobol' sensitivity indices , 2012, TOMC.

[11]  Jeremy E. Oakley,et al.  Managing structural uncertainty in health economic decision models: a discrepancy approach , 2012 .

[12]  Andrea Saltelli,et al.  From screening to quantitative sensitivity analysis. A unified approach , 2011, Comput. Phys. Commun..

[13]  S. Tarantola,et al.  Moment Independent Importance Measures: New Results and Analytical Test Cases , 2011, Risk analysis : an official publication of the Society for Risk Analysis.

[14]  Stoyan V. Stoyanov,et al.  A Probability Metrics Approach to Financial Risk Measures: Rachev/A Probability Metrics Approach to Financial Risk Measures , 2011 .

[15]  Alan Brennan,et al.  Simulation sample sizes for Monte Carlo partial EVPI calculations. , 2010, Journal of health economics.

[16]  Elmar Plischke,et al.  An effective algorithm for computing global sensitivity indices (EASI) , 2010, Reliab. Eng. Syst. Saf..

[17]  Karl Pearson,et al.  On the General Theory of Skew Correlation and Non-Linear Regression , 2010 .

[18]  Paola Annoni,et al.  Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index , 2010, Comput. Phys. Commun..

[19]  Mark Tygert,et al.  Statistical tests for whether a given set of independent, identically distributed draws comes from a specified probability density , 2010, Proceedings of the National Academy of Sciences.

[20]  Jim Berger,et al.  Special Issue on Computer Modeling , 2009, Technometrics.

[21]  Jeremy E. Oakley,et al.  Decision-Theoretic Sensitivity Analysis for Complex Computer Models , 2009, Technometrics.

[22]  Alfio Marazzi,et al.  Robust Response Transformations Based on Optimal Prediction , 2009 .

[23]  R. Cook,et al.  Principal fitted components for dimension reduction in regression , 2008, 0906.3953.

[24]  Max D. Morris,et al.  Using Orthogonal Arrays in the Sensitivity Analysis of Computer Models , 2008, Technometrics.

[25]  Saltelli Andrea,et al.  Global Sensitivity Analysis: The Primer , 2008 .

[26]  Emanuele Borgonovo,et al.  A new uncertainty importance measure , 2007, Reliab. Eng. Syst. Saf..

[27]  Hao Helen Zhang,et al.  Component selection and smoothing in multivariate nonparametric regression , 2006, math/0702659.

[28]  Igor Vajda,et al.  On Divergences and Informations in Statistics and Information Theory , 2006, IEEE Transactions on Information Theory.

[29]  Ronald L Iman,et al.  Sensitivity Analysis for Computer Model Projections of Hurricane Losses , 2005, Risk analysis : an official publication of the Society for Risk Analysis.

[30]  Thomas W. Lucas,et al.  State-of-the-Art Review: A User's Guide to the Brave New World of Designing Simulation Experiments , 2005, INFORMS J. Comput..

[31]  Xiao-Li Meng,et al.  From Unit Root to Stein’s Estimator to Fisher’s k Statistics: If You Have a Moment, I Can Tell You More , 2005 .

[32]  I. Csiszár,et al.  Information Theory and Statistics: A Tutorial , 2004, Found. Trends Commun. Inf. Theory.

[33]  A. O'Hagan,et al.  Probabilistic sensitivity analysis of complex models: a Bayesian approach , 2004 .

[34]  Alison L Gibbs,et al.  On Choosing and Bounding Probability Metrics , 2002, math/0209021.

[35]  A. Saltelli,et al.  On the Relative Importance of Input Factors in Mathematical Models , 2002 .

[36]  A. Saltelli,et al.  Making best use of model evaluations to compute sensitivity indices , 2002 .

[37]  I. Sobola,et al.  Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates , 2001 .

[38]  Robert F. Bordley,et al.  Decision analysis using targets instead of utility functions , 2000 .

[39]  J. Fine,et al.  Risk Assessment via a Robust Probit Model, with Application to Toxicology , 2000 .

[40]  H. Rabitz,et al.  General foundations of high‐dimensional model representations , 1999 .

[41]  M. McAleer,et al.  Tests of linear and logarithmic transformations for integrated processes , 1999 .

[42]  Lfp Pascal Etman,et al.  Design and analysis of computer experiments , 1998 .

[43]  F. J. Hickernell.  Quadrature Error Bounds with Applications to Lattice Rules , 1997 .

[44]  A. Saltelli,et al.  Importance measures in global sensitivity analysis of nonlinear models , 1996 .

[45]  E. Soofi.  Capturing the Intangible Concept of Information , 1994 .

[46]  Max D. Morris,et al.  Factorial sampling plans for preliminary computational experiments , 1991 .

[47]  S. Hora,et al.  A Robust Measure of Uncertainty Importance for Use in Fault Tree System Analysis , 1990 .

[48]  David Draper,et al.  Rank-Based Robust Analysis of Linear Models. I. Exposition and Review , 1988 .

[49]  L. Devroye,et al.  Nonparametric density estimation : the L[1] view , 1987 .

[50]  A. Takemura.  Tensor Analysis of ANOVA Decomposition , 1983 .

[51]  J. H. Schuenemeyer,et al.  A Modified Kolmogorov-Smirnov Test Sensitive to Tail Alternatives , 1983 .

[52]  R. Iman,et al.  Rank Transformations as a Bridge between Parametric and Nonparametric Statistics , 1981 .

[53]  B. Efron,et al.  The Jackknife Estimate of Variance , 1981 .

[54]  D. Groggel.  Practical Nonparametric Statistics , 1972, Technometrics.

[55]  D. Cox,et al.  An Analysis of Transformations , 1964 .

[56]  Egon S. Pearson,et al.  Comparison of tests for randomness of points on a line , 1963 .

[57]  W. Hoeffding.  A Class of Statistics with Asymptotically Normal Distribution , 1948 .

[58]  H. Scheffé.  A Useful Convergence Theorem for Probability Distributions , 1947 .

[59]  J. Schreiber.  Foundations Of Statistics , 2016 .

[60]  Bertrand Iooss,et al.  Sensitivity analysis of computer experiments , 2015 .

[61]  Elena Deza,et al.  Dictionary of distances , 2006 .

[62]  A. Owen.  THE DIMENSION DISTRIBUTION AND QUADRATURE TEST FUNCTIONS , 2003 .

[63]  Stefano Tarantola,et al.  Sensitivity Analysis as an Ingredient of Modeling , 2000 .

[64]  Andreas Griewank,et al.  Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition , 2000, Frontiers in applied mathematics.

[65]  I. Sobol,et al.  About the use of rank transformation in sensitivity analysis of model output , 1995 .

[66]  Ilya M. Sobol,et al.  Sensitivity Estimates for Nonlinear Mathematical Models , 1993 .

[67]  James V. Bondar,et al.  Mathematical theory of statistics , 1985 .

[68]  N. Kuiper.  Tests concerning random points on a circle , 1960 .

[69]  T. W. Anderson,et al.  Asymptotic Theory of Certain "Goodness of Fit" Criteria Based on Stochastic Processes , 1952 .