Accounting for input model and parameter uncertainty in simulation

Taking into account input-model, input-parameter, and stochastic uncertainties inherent in many simulations, our Bayesian approach to input modeling yields valid point and confidence-interval estimators for a selected posterior mean response. Exploiting prior information to specify the prior plausibility of each candidate input model and to construct prior distributions on the model's parameters, we combine this information with the likelihood function of sample data to compute posterior model probabilities and parameter distributions. Our Bayesian Simulation Replication Algorithm involves: (a) estimating parameter uncertainty by sampling from the posterior parameter distributions on selected runs; (b) estimating stochastic uncertainty by multiple independent replications of those runs; and (c) estimating model uncertainty by weighting the results of (a) and (b) using the corresponding posterior model probabilities. We allocate runs in (a) and (b) to minimize final estimator variance subject to a computing-budget constraint. An experimental performance evaluation demonstrates the advantages of this approach.

[1]  Alan E. Gelfand,et al.  Model Determination using sampling-based methods , 1996 .

[2]  David Draper,et al.  Assessment and Propagation of Model Uncertainty , 2011 .

[3]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[4]  Faker Zouaoui Accounting for Input Uncertainty in Discrete-Event Simulation , 2001 .

[5]  R. Cheng,et al.  Sensitivity of computer simulation experiments to errors in input data , 1997 .

[6]  Alan J. Miller,et al.  Subset Selection in Regression , 1991 .

[7]  James R. Wilson,et al.  Accounting for parameter uncertainty in simulation input modeling , 2001, Proceeding of the 2001 Winter Simulation Conference (Cat. No.01CH37304).

[8]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[9]  J. Berger,et al.  The Intrinsic Bayes Factor for Model Selection and Prediction , 1996 .

[10]  James R. Wilson,et al.  Accounting for Parameter Uncertainty in Simulation Input Modeling , 2003 .

[11]  S. T. Buckland,et al.  An Introduction to the Bootstrap. , 1994 .

[12]  Adrian E. Raftery,et al.  Bayesian model averaging: a tutorial (with comments by M. Clyde, David Draper and E. I. George, and a rejoinder by the authors , 1999 .

[13]  P. Davies,et al.  Kendall's Advanced Theory of Statistics. Volume 1. Distribution Theory , 1988 .

[14]  M. Kendall,et al.  Kendall's advanced theory of statistics , 1995 .

[15]  J. Berger,et al.  Testing Precise Hypotheses , 1987 .

[16]  Averill M. Law,et al.  How the Expertfit distribution-fitting software can make your simulation models more valid , 2003, Proceedings of the 2003 Winter Simulation Conference, 2003..

[17]  Stephen E. Chick Steps to implement Bayesian input distribution selection , 1999, WSC '99.

[18]  Alan J. Miller Subset Selection in Regression , 1992 .

[19]  Charles J. Geyer,et al.  Practical Markov Chain Monte Carlo , 1992 .

[20]  S. Wittevrongel,et al.  Queueing Systems , 2019, Introduction to Stochastic Processes and Simulation.

[21]  B. L. Welch On Linear Combinations of Several Variances , 1956 .

[22]  Satterthwaite Fe An approximate distribution of estimates of variance components. , 1946 .

[23]  F. E. Satterthwaite An approximate distribution of estimates of variance components. , 1946, Biometrics.

[24]  Peter Green,et al.  Markov chain Monte Carlo in Practice , 1996 .

[25]  Bo Thiesson,et al.  Diagnostic systems by model selection: a case study , 1994 .

[26]  M. Kendall,et al.  Kendall's Advanced Theory of Statistics: Volume 1 Distribution Theory , 1987 .

[27]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[28]  D. Madigan,et al.  Model Selection and Accounting for Model Uncertainty in Graphical Models Using Occam's Window , 1994 .

[29]  A. Law,et al.  Relative width sequential confidence intervals for the mean , 1981 .

[30]  G. J. A. Stern,et al.  Queueing Systems, Volume 2: Computer Applications , 1976 .

[31]  P. Rao Variance components estimation : mixed models, methodologies and applications , 1997 .

[32]  Averill M. Law,et al.  How the ExpertFit distribution-fitting package can make your simulation models more valid , 2000, 2000 Winter Simulation Conference Proceedings (Cat. No.00CH37165).

[33]  Purushottam W. Laud,et al.  A Predictive Approach to the Analysis of Designed Experiments , 1994 .