Multivariate Input Uncertainty in Output Analysis for Stochastic Simulation

When we use simulations to estimate the performance of stochastic systems, the simulation is often driven by input models estimated from finite real-world data. A complete statistical characterization of system performance estimates requires quantifying both input model and simulation estimation errors. The components of input models in many complex systems could be dependent. In this paper, we represent the distribution of a random vector by its marginal distributions and a dependence measure: either product-moment or Spearman rank correlations. To quantify the impact from dependent input model and simulation estimation errors on system performance estimates, we propose a metamodel-assisted bootstrap framework that is applicable to cases when the parametric family of multivariate input distributions is known or unknown. In either case, we first characterize the input models by their moments that are estimated using real-world data. Then, we employ the bootstrap to quantify the input estimation error, and an equation-based stochastic kriging metamodel to propagate the input uncertainty to the output mean, which can also reduce the influence of simulation estimation error due to output variability. Asymptotic analysis provides theoretical support for our approach, while an empirical study demonstrates that it has good finite-sample performance.

[1]  M. Rosenlicht Introduction to Analysis , 1970 .

[2]  Szu Hui Ng,et al.  A study on the effects of parameter estimation on kriging model's prediction error in stochastic simulations , 2009, Proceedings of the 2009 Winter Simulation Conference (WSC).

[3]  R. R. Barton,et al.  Tutorial: Input uncertainty in outout analysis , 2012, Proceedings Title: Proceedings of the 2012 Winter Simulation Conference (WSC).

[4]  R. Cheng,et al.  Sensitivity of computer simulation experiments to errors in input data , 1997 .

[5]  L NelsonBarry,et al.  Multivariate Input Uncertainty in Output Analysis for Stochastic Simulation , 2016 .

[6]  A. V. D. Vaart,et al.  Asymptotic Statistics: U -Statistics , 1998 .

[7]  Joseph L. Hammond,et al.  Generation of Pseudorandom Numbers with Specified Univariate Distributions and Correlation Coefficients , 1975, IEEE Transactions on Systems, Man, and Cybernetics.

[8]  J. Norris Appendix: probability and measure , 1997 .

[9]  Russell R. Barton Input uncertainty in outout analysis , 2012, Winter Simulation Conference.

[10]  François Bachoc,et al.  Cross Validation and Maximum Likelihood estimations of hyper-parameters of Gaussian processes with model misspecification , 2013, Comput. Stat. Data Anal..

[11]  Bruce W. Schmeiser,et al.  Bivariate Gamma Random Vectors , 1982, Oper. Res..

[12]  Wei Xie,et al.  Quantifying Input Uncertainty via Simulation Confidence Intervals , 2014, INFORMS J. Comput..

[13]  Roger M. Cooke,et al.  Generating "dependent" quasi-random numbers , 2000, 2000 Winter Simulation Conference Proceedings (Cat. No.00CH37165).

[14]  Shane G. Henderson,et al.  Properties of the NORTA method in higher dimensions , 2002, Proceedings of the Winter Simulation Conference.

[15]  Russell R. Barton A MORE COMPLETE CHARACTERIZATION OF UNCERTAINTY : CAN IT BE DONE ? , 2007 .

[16]  Wei Xie,et al.  Statistical uncertainty analysis for stochastic simulation with dependent input models , 2014, Proceedings of the Winter Simulation Conference 2014.

[17]  Barry L. Nelson,et al.  Advanced tutorial: Input uncertainty quantification , 2014, Proceedings of the Winter Simulation Conference 2014.

[18]  Barry L. Nelson,et al.  Input uncertainty quantification: advanced tutorial , 2014, WSC '14.

[19]  Lee W. Schruben,et al.  Resampling methods for input modeling , 2001, Proceeding of the 2001 Winter Simulation Conference (Cat. No.01CH37304).

[20]  J. Mielniczuk,et al.  A new look at measuring dependence , 2010 .

[21]  Mark E. Johnson,et al.  Multivariate Statistical Simulation , 1989, International Encyclopedia of Statistical Science.

[22]  J. A. Cuesta-Albertos,et al.  Unconditional Glivenko-Cantelli-type theorems and weak laws of large numbers for bootstrap , 1996 .

[23]  Lee W. Schruben,et al.  Uniform and bootstrap resampling of empirical distributions , 1993, WSC '93.

[24]  Sondipon Adhikari,et al.  A Critical Assessment of Kriging Model Variants for High-Fidelity Uncertainty Quantification in Dynamics of composite Shells , 2016, Archives of Computational Methods in Engineering.

[25]  Jerome Sacks,et al.  Choosing the Sample Size of a Computer Experiment: A Practical Guide , 2009, Technometrics.

[26]  Xi Chen,et al.  The effects of common random numbers on stochastic kriging metamodels , 2012, TOMC.

[27]  N. Higham Computing the nearest correlation matrix—a problem from finance , 2002 .

[28]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[29]  R. Clemen,et al.  Correlations and Copulas for Decision and Risk Analysis , 1999 .

[30]  On the Estimation of Parameters of Variograms of Spatial Stationary Isotropic Random Processes , 2012 .

[31]  Wei Xie,et al.  A Bayesian Framework for Quantifying Uncertainty in Stochastic Simulation , 2014, Oper. Res..

[32]  Shane G. Henderson,et al.  Chessboard Distributions and Random Vectors with Specified Marginals and Covariance Matrix , 2002, Oper. Res..

[33]  Bruce W. Schmeiser,et al.  Graphical Methods for Evaluating and Comparing Confidence-Interval Procedures , 1990, Oper. Res..

[34]  Mark E. Johnson Multivariate Statistical Simulation: Johnson/Multivariate , 1987 .

[35]  P. Hall Rate of convergence in bootstrap approximations , 1988 .

[36]  Wei Xie,et al.  The influence of correlation functions on stochastic kriging metamodels , 2010, Proceedings of the 2010 Winter Simulation Conference.

[37]  Barry L. Nelson,et al.  Foundations and Methods of Stochastic Simulation: A First Course , 2013 .

[38]  J. Shao,et al.  The jackknife and bootstrap , 1996 .

[39]  Barry L. Nelson,et al.  Stochastic kriging for simulation metamodeling , 2008, 2008 Winter Simulation Conference.

[40]  Russell R. Barton,et al.  Statistical Uncertainty Analysis for Stochastic Simulation , 2014 .

[41]  Canan G. Corlu,et al.  Accounting for Parameter Uncertainty in Large-Scale Stochastic Simulations with Correlated Inputs , 2011, Oper. Res..