ASAP3: a batch means procedure for steady-state simulation analysis

We introduce ASAP3, a refinement of the batch means algorithms ASAP and ASAP2, that delivers point and confidence-interval estimators for the expected response of a steady-state simulation. ASAP3 is a sequential procedure designed to produce a confidence-interval estimator that satisfies user-specified requirements on absolute or relative precision as well as coverage probability. ASAP3 operates as follows: the batch size is progressively increased until the batch means pass the Shapiro-Wilk test for multivariate normality; and then ASAP3 fits a first-order autoregressive (AR(1)) time series model to the batch means. If necessary, the batch size is further increased until the autoregressive parameter in the AR(1) model does not significantly exceed 0.8. Next, ASAP3 computes the terms of an inverse Cornish-Fisher expansion for the classical batch means t-ratio based on the AR(1) parameter estimates; and finally ASAP3 delivers a correlation-adjusted confidence interval based on this expansion. Regarding not only conformance to the precision and coverage-probability requirements but also the mean and variance of the half-length of the delivered confidence interval, ASAP3 compared favorably to other batch means procedures (namely, ABATCH, ASAP, ASAP2, and LBATCH) in an extensive experimental performance evaluation.

[1]  Natalie M. Steiger,et al.  Output analysis: ASAP2: an improved batch means procedure for simulation output analysis , 2002, WSC '02.

[2]  J. Royston.  The W Test for Normality , 1982 .

[3]  Chris Chatfield,et al.  Introduction to Statistical Time Series. , 1976 .

[4]  James R. Wilson,et al.  Validation of Simulation Analysis Methods for the Schruben-Margolin Correlation-Induction Strategy , 1992, Oper. Res..

[5]  F. E. Satterthwaite.  An approximate distribution of estimates of variance components. , 1946, Biometrics.

[6]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[7]  George S. Fishman,et al.  An Implementation of the Batch Means Method , 1997, INFORMS J. Comput..

[8]  J. Royston.  An Extension of Shapiro and Wilk's W Test for Normality to Large Samples , 1982 .

[9]  J. P. Royston,et al.  Algorithm AS 181: The W Test for Normality , 1982 .

[10]  James R. Wilson,et al.  Experimental performance evaluation of batch means procedures for simulation output analysis , 2000, 2000 Winter Simulation Conference Proceedings (Cat. No.00CH37165).

[11]  Michael A. Malcolm,et al.  Computer methods for mathematical computations , 1977 .

[12]  James R. Wilson,et al.  Improved batching for confidence interval construction in steady-state simulation , 1999, WSC '99.

[13]  W. Fuller,et al.  Introduction to Statistical Time Series (2nd ed.) , 1997 .

[14]  David Goldsman,et al.  To batch or not to batch? , 2004, TOMC.

[15]  David Goldsman,et al.  Spaced batch means , 1991, Oper. Res. Lett..

[16]  George S. Fishman,et al.  LABATCH.2 for Analyzing Sample Path Data , 1998 .

[17]  Emily K. Lada,et al.  Simulation output analysis: a wavelet-based spectral method for steady-state simulation analysis , 2003, WSC '03.

[18]  S. R. Searle,et al.  Matrix Algebra Useful for Statistics , 1982 .

[19]  G. Box.  Some Theorems on Quadratic Forms Applied in the Study of Analysis of Variance Problems, I. Effect of Inequality of Variance in the One-Way Classification , 1954 .

[20]  Satterthwaite Fe.  An approximate distribution of estimates of variance components. , 1946 .

[21]  Andrés Suárez-González,et al.  New simulation output analysis techniques: a batch means procedure for mean value estimation of processes exhibiting long range dependence , 2002, WSC '02.

[22]  H. Robbins,et al.  ON THE ASYMPTOTIC THEORY OF FIXED-WIDTH SEQUENTIAL CONFIDENCE INTERVALS FOR THE MEAN. , 1965 .

[23]  Takeshi Amemiya,et al.  The Effect of Aggregation on Prediction in the Autoregressive Model , 1972 .

[24]  Barry L. Nelson,et al.  Statistical Analysis of Simulation Results , 2007 .

[25]  Christos Alexopoulos,et al.  Output Data Analysis , 2007 .

[26]  Abel M. Rodrigues.  Matrix Algebra Useful for Statistics , 2007 .

[27]  A. Afifi,et al.  On Tests for Multivariate Normality , 1973 .

[28]  F. E. Satterthwaite.  Synthesis of variance , 1941 .

[29]  N. L. Johnson,et al.  Continuous Univariate Distributions. , 1995 .

[30]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[31]  D. Goldsman,et al.  ASAP2: an improved batch means procedure for simulation output analysis , 2002, Proceedings of the Winter Simulation Conference.

[32]  G. M. Jenkins,et al.  An angular transformation for the serial correlation coefficient , 1954 .

[33]  B. L. Welch.  On Linear Combinations of Several Variances , 1956 .

[34]  Bruce W. Schmeiser,et al.  Properties of batch means from stationary ARMA time series , 1987 .

[35]  Emily K. Lada,et al.  Performance of a Wavelet-Based Spectral Procedure for Steady-State Simulation Analysis , 2007, INFORMS J. Comput..

[36]  M. Kendall,et al.  Kendall's Advanced Theory of Statistics: Volume 1 Distribution Theory , 1987 .

[37]  M SteigerNatalie,et al.  ASAP3: a batch means procedure for steady-state simulation analysis , 2005 .

[38]  P. Davies,et al.  Kendall's Advanced Theory of Statistics. Volume 1. Distribution Theory , 1988 .

[39]  Emily K. Lada,et al.  Performance evaluation of a wavelet-based spectral method for steady-state simulation analysis , 2004, Proceedings of the 2004 Winter Simulation Conference, 2004..

[40]  E.K. Lada,et al.  A wavelet-based spectral method for steady-state simulation analysis , 2003, Proceedings of the 2003 Winter Simulation Conference, 2003..

[41]  M. Fernandez-Veiga,et al.  A batch means procedure for mean value estimation of processes exhibiting long range dependence , 2002, Proceedings of the Winter Simulation Conference.

[42]  J. P. Secrétan,et al.  Der Saccus endolymphaticus bei Entzündungsprozessen , 1944 .

[43]  P. Bickel,et al.  Mathematical Statistics: Basic Ideas and Selected Topics , 1977 .

[44]  A. Nádas.  An Extension of a Theorem of Chow and Robbins on Sequential Confidence Intervals for the Mean , 1969 .

[45]  Z. Cheridjian.  Examen au point de vue osteocytaire d’une mâchoire inférieure atteinte de nécrose au Phosphore , 1948 .

[46]  Ronald L. Wasserstein,et al.  Monte Carlo: Concepts, Algorithms, and Applications , 1997 .

[47]  F. Grund.  Forsythe, G. E. / Malcolm, M. A. / Moler, C. B., Computer Methods for Mathematical Computations. Englewood Cliffs, New Jersey 07632. Prentice Hall, Inc., 1977. XI, 259 S , 1979 .

[48]  James R. Wilson,et al.  Convergence Properties of the Batch Means Method for Simulation Output Analysis , 2001, INFORMS J. Comput..