Steady-state simulation of queueing processes: survey of problems and solutions

For years computer-based stochastic simulation has been a commonly used tool in the performance evaluation of various systems. Unfortunately, the results of simulation studies quite often have little credibility, since they are presented without regard to their random nature and the need for proper statistical analysis of simulation output data. This paper discusses the main factors that can affect the accuracy of stochastic simulations designed to give insight into the steady-state behavior of queuing processes. The problems of correctly starting and stopping such simulation experiments to obtain the required statistical accuracy of the results are addressed. In this survey of possible solutions, the emphasis is put on possible applications in the sequential analysis of output data, which adaptively decides about continuing a simulation experiment until the required accuracy of results is reached. A suitable solution for deciding upon the starting point of a steady-state analysis and two techniques for obtaining the final simulation results to a required level of accuracy are presented, together with pseudocode implementations.

[1]  G. S. Fishman Grouping Observations in Digital Simulation , 1978 .

[2]  Young Hwan Park,et al.  A statistical expert system for simulation analysis , 1989, Simul..

[3]  Devendra Kumar A Novel Approach to Sequential Simulation , 1986, IEEE Software.

[4]  Russell C. H. Cheng,et al.  Variance reduction methods , 1986, WSC '86.

[5]  Wolfgang Kreutzer,et al.  System simulation programming styles and languages , 1986 .

[6]  K. Mani Chandy,et al.  Asynchronous distributed simulation via a sequence of parallel computations , 1981, CACM.

[7]  Lee W. Schruben,et al.  Detecting Initialization Bias in Simulation Output , 1982, Oper. Res..

[8]  G. Watson,et al.  Computer simulation , 1988 .

[9]  A. A. Crane,et al.  An introduction to the regenerative method for simulation analysis , 1977 .

[10]  N. Adam Achieving a Confidence Interval for Parameters Estimated by Simulation , 1983 .

[11]  Wolfgang Kreutzer,et al.  POSE: a Process-Oriented Simulation Environment embedded in SCHEME , 1988, Simul..

[12]  J. W. Tukey,et al.  The Measurement of Power Spectra from the Point of View of Communications Engineering , 1958 .

[13]  Jack P. C. Kleijnen,et al.  Testing independence of simulation subruns: a note on the power of the Von Neumann test , 1982 .

[14]  Lee W. Schruben,et al.  An interactive run length control for simulation on PCs , 1986, WSC '86.

[15]  Ward Whitt,et al.  Indirect Estimation Via L = λW , 1989, Oper. Res..

[16]  J. George Shanthikumar,et al.  A Unifying View of Hybrid Simulation/Analytic Models and Modeling , 1983, Oper. Res..

[17]  Adam Wolisz,et al.  Increasing the Efficiency of Multiple Access Protocol Simulation , 1987, Kommunikation in Verteilten Systemen.

[18]  P. Heidelberger,et al.  Adaptive spectral methods for simulation output analysis , 1981 .

[19]  Lee W. Schruben Overview of standardized time series , 1985, WSC '85.

[20]  D. G. Watts,et al.  Spectral analysis and its applications , 1968 .

[21]  Linda Weiser Friedman,et al.  Multivariate simulation output analysis: Past, present, and future , 1984, WSC '84.

[22]  Averill M. Law,et al.  A new approach for dealing with the startup problem in discrete event simulation , 1983 .

[23]  J. E. Glynn,et al.  Numerical Recipes: The Art of Scientific Computing , 1989 .

[24]  Averill M. Law,et al.  Confidence Intervals for Steady-State Simulations II: A Survey of Sequential Procedures , 1982 .

[25]  Emily Roth,et al.  A relaxation time approach for reducing initialization bias in simulation , 1985, ANSS '85.

[26]  J. Neumann Distribution of the Ratio of the Mean Square Successive Difference to the Variance , 1941 .

[27]  Paul F. Reynolds A spectrum of options for parallel simulation , 1988, WSC '88.

[28]  David Goldsman,et al.  A comparison between standardized time series and overlapping batched means , 1985, WSC '85.

[29]  T. Amemiya Generalized Least Squares with an Estimated Autocovariance Matrix , 1973 .

[30]  D. Hand Statistical expert systems: necessary attributes , 1985 .

[31]  Victor S. Frost,et al.  Some new efficient techniques for the simulation of computer communications networks , 1988, IEEE J. Sel. Areas Commun..

[32]  Averill M. Law,et al.  An Analytical Evaluation of Alternative Strategies in Steady-State Simulation , 1984, Oper. Res..

[33]  L. Schruben,et al.  Asymptotic Properties of Some Confidence Interval Estimators for Simulation Output , 1984 .

[34]  James R. Wilson,et al.  Evaluation of startup policies in simulation experiments , 1978 .

[35]  Richard W. Hamming,et al.  Numerical Methods for Scientists and Engineers , 1963 .

[36]  Jean Walrand,et al.  A quick simulation method for excessive backlogs in networks of queues , 1989 .

[37]  Carl M. Harris,et al.  Fundamentals of queueing theory , 1975 .

[38]  D. Daley The serial correlation coefficients of waiting times in a stationary single server queue , 1968, Journal of the Australian Mathematical Society.

[39]  George S. Fishman,et al.  Solution of Large Networks by Matrix Methods , 1976, IEEE Transactions on Systems, Man, and Cybernetics.

[40]  Reuven R. Levary,et al.  Validation of simulation models using decision theory , 1987, Simul..

[41]  George S. Fishman,et al.  Problems in the statistical analysis of simulation experiments: the comparision of means and the length of sample records , 1967, CACM.

[42]  Robert E. Shannon,et al.  Minimum mean-squared-error estimators for simulation experiments , 1981, CACM.

[43]  Do Le Minh Simulating GI/G/k Queues in Heavy Traffic , 1987 .

[44]  W. Whitt,et al.  Transient behavior of regulated Brownian motion, I: Starting at the origin , 1987, Advances in Applied Probability.

[45]  Richard W. Hamming,et al.  Numerical Methods for Scientists and Engineers , 1962 .

[46]  Steven D. Duket,et al.  Examination of simulation output using spectral methods , 1978 .

[47]  Bruce W. Schmeiser,et al.  Batch Size Effects in the Analysis of Simulation Output , 1982, Oper. Res..

[48]  Charles W. Beall A regression technique for determining steady state conditions in time series simulations , 1982, Winter Simulation Conference.

[49]  Averill M. Law,et al.  Simulation Modeling and Analysis , 1982 .

[50]  K. Pawlikowski,et al.  Simulation studies of mixed traffic on satellite channels using TDMA-reservation protocol , 1989, Eighth Annual International Phoenix Conference on Computers and Communications. 1989 Conference Proceedings.

[51]  Thomas J. Schriber,et al.  Simulation Using GPSS , 1974 .

[52]  Geoffrey Gordon,et al.  System Simulation , 1970 .

[53]  Wolfgang Kreutzer,et al.  System simulation , 1986 .

[54]  George S. Fishman,et al.  Statistical Analysis for Queueing Simulations , 1973 .

[55]  Averill M. Law,et al.  Feature Article - Statistical Analysis of Simulation Output Data , 1983, Oper. Res..

[56]  Joseph J. Talavage,et al.  A hybrid paradigm for modeling of complex systems , 1987, Simul..

[57]  Philip Heidelberger,et al.  Quantile Estimation in Dependent Sequences , 1984, Oper. Res..

[58]  Lee W. Schruben Control of initialization bias in multivariate simulation response , 1981, CACM.

[59]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[60]  John Harling,et al.  Simulation Techniques in Operations Research---A Review , 1958 .

[61]  Terry Williams,et al.  Probability and Statistics with Reliability, Queueing and Computer Science Applications , 1983 .

[62]  Alan Weiss,et al.  Sensitivity analysis via likelihood ratios , 1986, WSC '86.

[63]  Thomas J. Schriber,et al.  A conceptual framework for research in the analysis of simulation output , 1981, CACM.

[64]  A. DeCegama Parallel processing simulation of large computer networks , 1987 .

[65]  Susan L. Solomon Simulation of waiting-line systems , 1983 .

[66]  James R. Emshoff,et al.  Design and use of computer simulation models , 1970 .

[67]  Kimmo E. E. Raatikainen,et al.  Simultaneous estimation of several percentiles , 1987, Simul..

[68]  Bernard P. Zeigler,et al.  Artificial intelligence in modelling and simulation: Directions to explore , 1987, Simul..

[69]  J. Kiefer,et al.  On the theory of queues with many servers , 1955 .

[70]  George S. Fishman,et al.  The Analysis of Simulation-Generated Time Series , 1967 .

[71]  C. J. Ancker,et al.  Evaluation of commonly used rules for detecting “steady state” in computer simulation , 1978 .

[72]  S. Shapiro,et al.  An Analysis of Variance Test for Normality (Complete Samples) , 1965 .

[73]  Donald L. Iglehart,et al.  Large-sample theory for standardized time series: an overview , 1985, WSC '85.

[74]  Bernard P. Zeigler,et al.  Performance evaluation of hierarchical distributed simulators , 1985, WSC '85.

[75]  J. R. Walters,et al.  Statistical Techniques in Simulation (2 vols.) , 1976 .

[76]  James R. Wilson,et al.  A survey of research on the simulation startup problem , 1978 .

[77]  Edward D. Lazowska,et al.  Parallel simulation of queueing networks: limitations and potentials , 1989, SIGMETRICS '89.

[78]  Amedeo R. Odoni,et al.  An Empirical Investigation of the Transient Behavior of Stationary Queueing Systems , 1983, Oper. Res..

[79]  David Goldsman,et al.  Ranking and selection in simulation , 1983, WSC '83.

[80]  Robert M. O'Keefe,et al.  Simulation and expert systems- A taxonomy and some examples , 1986 .

[81]  M. Bartlett On the Theoretical Specification and Sampling Properties of Autocorrelated Time‐Series , 1946 .

[82]  Linus Schrage,et al.  A guide to simulation , 1983 .

[83]  Jack P. C. Kleijnen,et al.  The role of statistical methodology in simulation , 1978, SIML.

[84]  K. Pearson Biometrika , 1902, The American Naturalist.

[85]  D. Brillinger Estimation of the mean of a stationary time series by sampling , 1973, Journal of Applied Probability.

[86]  Emily Roth A relaxation time heuristic for reducing initialization bias in simulation of M/Ek/1 queueing systems , 1985, WSC '85.

[87]  David Goldsman,et al.  Tutorial on indifference-zone normal means ranking and selection procedures , 1986, WSC '86.

[88]  David M. Nicol Parallel discrete-event simulation of FCFS stochastic queueing networks , 1988, PPoPP 1988.

[89]  H. T. Mouftah,et al.  Computer-aided modeling, analysis, and design of communication networks , 1988, IEEE J. Sel. Areas Commun..

[90]  Lee W. Schruben,et al.  The initial transient in steady state simulation (Panel Discussion) , 1981, WSC '81.

[91]  Averill M. Law,et al.  A Sequential Procedure for Determining the Length of a Steady-State Simulation , 1979, Oper. Res..

[92]  Bruce W. Schmeiser,et al.  Minimal-MSE linear combinations of variance estimators of the sample mean , 1988, WSC '88.

[93]  James R. Wilson,et al.  Variance reduction: The current state , 1983 .

[94]  Robert G. Sargent,et al.  The use of graphical models in model validation , 1986, WSC '86.

[95]  Stephen S. Lavenberg,et al.  Statistical Results on Control Variables with Application to Queueing Network Simulation , 1982, Oper. Res..

[96]  P. Billingsley,et al.  Convergence of Probability Measures , 1969 .

[97]  Andrew F. Seila,et al.  Multivariate inference in stationary simulation using batch means , 1987, WSC '87.

[98]  James R. Wilson,et al.  Estimation procedures based on control variates with known covariance matrix , 1987, WSC '87.

[99]  N. Blomqvist The covariance function of the M/G/1 queuing system , 1967 .

[100]  M. Geisler The Sizes of Simulation Samples Required to Compute Certain Inventory Characteristics with Stated Precision and Confidence , 1964 .

[101]  Verna Knapp,et al.  The Smalltalk simulation environment , 1986, WSC '86.

[102]  N. Blomqvist On the transient behaviour of the GI/G/1 waiting-times , 1970 .

[103]  William L. Maxwell,et al.  Some Problems of Digital Systems Simulation , 1959 .

[104]  E. Parzen Mathematical Considerations in the Estimation of Spectra , 1961 .

[105]  Thomas J. Schriber,et al.  Interactive analysis of simulation output by the method of batch means , 1979, WSC '79.

[106]  Charles H. Sauer,et al.  Queueing Network Simulations of Computer Communication , 1984, IEEE J. Sel. Areas Commun..

[107]  Heimo H. Adelsberger,et al.  Expert systems and simulation , 1985 .

[108]  Jorge Haddock An expert system framework based on a simulation generator , 1987, Simul..

[109]  Ward Whitt,et al.  Transient behavior of regulated Brownian motion, II: Non-zero initial conditions , 1987 .

[110]  Brian W. Unger Distributed simulation , 1988, WSC '88.

[111]  Stephen S. Lavenberg,et al.  A Perspective on the Use of Control Variables to Increase the Efficiency of Monte Carlo Simulations , 1981 .

[112]  박영흥 A Statistical Expert System for Simulation Analysis , 1991 .

[113]  Kishor S. Trivedi Probability and Statistics with Reliability, Queuing, and Computer Science Applications , 1984 .

[114]  Andrew F. Seila,et al.  Multivariate estimation in regenerative simulation , 1982, Oper. Res. Lett..

[115]  Lee W. Schruben Using simulation to solve problems: a tutorial on the analysis of simulation output , 1987, WSC '87.

[116]  J. Kleijnen Statistical tools for simulation practitioners , 1986 .

[117]  Murali Krishnamurthi,et al.  Two approaches to the implementation of a distributed simulation system , 1985, WSC '85.

[118]  J. P. C. Blanc The Transient Behaviour of Networks with Infinite Server Nodes , 1984, Performance.

[119]  Troy Henson Artificial intelligence and simulation : the diversity of applications : proceedings of the SCS Multiconference on Artificial Intelligence and Simulation : the diversity of applications, 3-5 February 1988, San Diego, California , 1988 .

[120]  Tuncer I. Ören,et al.  Quality assurance paradigms for artificial intelligence in modelling and simulation , 1987, Simul..

[121]  R. Reddy Epistemology of knowledge based simulation , 1987, Simul..

[122]  Christos G. Cassandras,et al.  Infinitesimal and finite perturbation analysis for queueing networks , 1982, 1982 21st IEEE Conference on Decision and Control.

[123]  Albert Madansky Technical Note - Optimal Initial Conditions for a Simulation Problem , 1976, Oper. Res..

[124]  G. C. Vansteenkiste,et al.  The impact of advanced information processing on simulation — An illustrative review , 1986 .

[125]  Averill M. Law Confidence intervals in discrete event simulation: A comparison of replication and batch means , 1977 .

[126]  Lee W. Shruben Control of initialization bias in multivariate simulation response , 1981 .

[127]  Halim Damerdji On strong consistency of the variance estimator , 1987, WSC '87.

[128]  Bruce W. Schmeiser,et al.  Overlapping batch means: something for nothing? , 1984, WSC '84.

[129]  Richard M. Fujimoto,et al.  Performance Measurements of Distributed Simulation Strategies. , 1987 .

[130]  Jack P. C. Kleijnen,et al.  Statistical Techniques in Simulation , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[131]  Gerald J. Hahn,et al.  More intelligent statistical software and statistical expert systems: future directions , 1985 .

[132]  Charles H. Sauer,et al.  The research queueing package: past, present, and future , 1982, AFIPS '82.

[133]  共立出版株式会社 コンピュータ・サイエンス : ACM computing surveys , 1978 .

[134]  Marie Cottrell,et al.  Large deviations and rare events in the study of stochastic algorithms , 1983 .

[135]  Robert E. Shannon Tests for the verification and validation of computer simulation models , 1981, WSC '81.

[136]  W. Whitt,et al.  Transient behavior of the M/M/1 queue via Laplace transforms , 1988, Advances in Applied Probability.

[137]  Bennett L. Fox Estimation and Simulation , 1978 .

[138]  Erhan Çinlar,et al.  Introduction to stochastic processes , 1974 .

[139]  Paul Bratley,et al.  A guide to simulation (2nd ed.) , 1986 .

[140]  Philip Heidelberger A VARIANCE REDUCTION TECHNIQUE THAT INCREASES REGENERATION FREQUENCY , 1979 .

[141]  Yu-Chi Ho,et al.  Performance evaluation and perturbation analysis of discrete event dynamic systems , 1987 .

[142]  James R. Wilson,et al.  The efficiency of control variates in multiresponse simulation , 1986 .

[143]  W. David Kelton,et al.  Transient exponential-Erlang queues and steady-state simulation , 1985, CACM.

[144]  Averill M. Law,et al.  The Transient Behavior of the M/M/s Queue, with Implications for Steady-State Simulation , 1985, Oper. Res..

[145]  A. F. Seila,et al.  A Batching Approach to Quantile Estimation in Regenerative Simulations , 1982 .

[146]  Ward Whitt,et al.  Transient behavior of the M/M/l queue: Starting at the origin , 1987, Queueing Syst. Theory Appl..

[147]  Edward A. MacNair An introduction to the Research Queueing Package , 1985, WSC '85.

[148]  Robert G. Sargent,et al.  Statistical analysis of simulation output data , 1976, SIML.

[149]  Keebom Kang,et al.  The correlation between mean and variance estimators , 1985, WSC '85.

[150]  Carl M. Harris,et al.  Fundamentals of queueing theory (2nd ed.). , 1985 .

[151]  John K. Jackman,et al.  A graphical methodology for simulating communication networks , 1988, IEEE Trans. Commun..

[152]  Reuven Y. Rubinstein,et al.  Sensitivity Analysis and Performance Extrapolation for Computer Simulation Models , 1989, Oper. Res..

[153]  F. L. Gunther,et al.  The Almost Regenerative Method for Stochastic System Simulations , 1980, Oper. Res..

[154]  Chin-Fu Ho,et al.  An algorithm for testing serial dependence of simulation output data , 1988, 1988 Winter Simulation Conference Proceedings.

[155]  Andrew F. Seila Multivariate estimation in simulation , 1983, WSC '83.

[156]  S. K. Park,et al.  Random number generators: good ones are hard to find , 1988, CACM.

[157]  Jayadev Misra,et al.  Distributed discrete-event simulation , 1986, CSUR.

[158]  William G. Bulgren Discrete system simulation , 1982 .

[159]  Lee W. Schruben,et al.  Confidence Interval Estimation Using Standardized Time Series , 1983, Oper. Res..

[160]  B L Nelson,et al.  Decomposition of Some Well-Known Variance Reduction Techniques. Revision. , 1985 .

[161]  Wei-Ning Yang,et al.  A Bonferroni selection procedure when using commom random numbers with unknown variances , 1986, WSC '86.

[162]  Lee W. Schruben,et al.  Optimal Tests for Initialization Bias in Simulation Output , 1983, Oper. Res..

[163]  Philip Heidelberger,et al.  A spectral method for confidence interval generation and run length control in simulations , 1981, CACM.

[164]  David M. Nicol,et al.  Parallel discrete-event simulation of FCFS stochastic queueing networks , 1988, PPEALS '88.

[165]  Richard W. Conway,et al.  Some Tactical Problems in Digital Simulation , 1963 .

[166]  Thomas J. Schriber,et al.  Interactive analysis of output from GPSS-based simulations , 1978, WSC '78.

[167]  Allen J. Scott,et al.  Applications of Queueing Theory , 1972 .

[168]  Averill M. Law,et al.  Confidence Intervals for Steady-State Simulations: I. A Survey of Fixed Sample Size Procedures , 1984, Oper. Res..

[169]  Robert M. O'Keefe,et al.  Visual Interactive Simulation — History, recent developments, and major issues , 1987, Simul..

[170]  Hershey H. Friedman,et al.  Comparing simulated alternatives using a distribution-free statistic with blocking by random number stream , 1986, Simul..

[171]  Averill M. Law Efficient estimators for simulated queueing systems , 1974 .

[172]  Philip Heidelberger Statistical analysis of parallel simulations , 1986, WSC '86.

[173]  James R. Wilson Variance Reduction Techniques for Digital Simulation , 1984 .

[174]  Michael Pidd,et al.  Computer Simulation in Management Science (3rd Edition) , 1998 .

[175]  W. David Kelton,et al.  Statistical design and analysis , 1986, WSC '86.

[176]  Donald L. Iglehart,et al.  Simulating Stable Stochastic Systems, VI: Quantile Estimation , 1976, JACM.

[177]  E. J. Hannan,et al.  Multiple time series , 1970 .

[178]  Vincent Hodgson,et al.  The Single Server Queue. , 1972 .

[179]  Peter D. Welch,et al.  On the relationship between batch means, overlapping means and spectral estimation , 1987, WSC '87.

[180]  L. Schruben A Coverage Function for Interval Estimators of Simulation Response , 1980 .

[181]  K. D. Tocher,et al.  The art of simulation , 1967 .

[182]  Bruce W. Schmeiser,et al.  Correlation among estimators of the variance of the sample mean , 1987, WSC '87.

[183]  Barry L. Nelson,et al.  Automated estimation and variance reduction for steady-state simulations , 1986, WSC '86.

[184]  James A. Payne,et al.  Introduction to Simulation: Programming Techniques and Methods Analysis , 1982 .

[185]  Y. Ho,et al.  Perturbation analysis and optimization of queueing networks , 1982, 1982 21st IEEE Conference on Decision and Control.

[186]  Richard A. Davis,et al.  Time Series: Theory and Methods , 2013 .

[187]  George S. Fishman Estimation in Multiserver Queuing Simulations , 1974, Oper. Res..

[188]  Kim Andersen Some steady-state problems in simulation , 1985, ANSS '85.

[189]  M LawAverill,et al.  Confidence Intervals for Steady-State Simulations II , 1982 .

[190]  Jack P. C. Kleijnen,et al.  Statistical Techniques in Simulation , 1977, IEEE Transactions on Systems, Man and Cybernetics.

[191]  Wei-Ning Yang,et al.  Multivariate estimation and variance reduction in terminating and steady-state simulation , 1988, WSC '88.

[192]  W. David Kelton,et al.  Initializing for bias reduction: some analytical considerations , 1988, WSC '88.

[193]  J. S. Dagpunar,et al.  Principles of Discrete Event Simulation , 1980 .

[194]  Reuven Y. Rubinstein,et al.  Efficiency of Multivariate Control Variates in Monte Carlo Simulation , 1985, Oper. Res..

[195]  R. Suri,et al.  Perturbation analysis gives strongly consistent sensitivity estimates for the M/G/ 1 queue , 1988 .

[196]  George S. Fishman,et al.  Achieving specific accuracy in simulation output analysis , 1977, CACM.

[197]  C. H. Sauer,et al.  Sequential stopping rules for the regenerative method of simulation , 1977 .

[198]  James R. Wilson,et al.  Using path control variates in activity network simulation , 1985, WSC '85.

[199]  Peter W. Glynn Coverage error for confidence intervals arising in simulation output analysis , 1982, Winter Simulation Conference.

[200]  David N. Nicol High performance parallelized discrete event simulation of stochastic queueing networks , 1988, WSC '88.

[201]  R. Cheng A Note on the Effect of Initial Conditions on a Simulation Run , 1976 .

[202]  James M. Calvin Covariance of regenerative mean and variance estimators for Markov chains , 1988, WSC '88.

[203]  George S. Fishman,et al.  Bias Considerations in Simulation Experiments , 1972, Oper. Res..

[204]  Joseph M. Sussman,et al.  Toward guidelines for designing experiments in queuing simulation , 1977 .

[205]  Philip Heidelberger,et al.  Regression-adjusted estimates for regenerative simulations, with graphics , 1981, CACM.

[206]  J.P.C. Blanc,et al.  The relaxation time of two queueing systems in series , 1983 .

[207]  George S. Fishman,et al.  Estimating Sample Size in Computing Simulation Experiments , 1971 .

[208]  A. Law Statistical Analysis of the Output Data from Terminating Simulations. , 1980 .

[209]  G. Wahba Automatic Smoothing of the Log Periodogram , 1980 .

[210]  Neil B. Marks,et al.  Further investigation into spectral analysis for confidence intervals in steady state simulations , 1981, WSC '81.

[211]  Philip Heidelberger,et al.  Simulation Run Length Control in the Presence of an Initial Transient , 1983, Oper. Res..

[212]  David W. Balmer Polishing the analysis of the statistical output of comparative simulation experiments , 1987, Simul..

[213]  P. Heidelberger,et al.  A Renewal Theoretic Approach to Bias Reduction in Regenerative Simulations , 1982 .

[214]  Keebom Kang,et al.  Large and small sample comparisons of various variance estimators , 1986, WSC '86.

[215]  Isi Mitrani Simulation techniques for discrete event systems , 1982, Cambridge computer science texts.

[216]  Rupert G. Miller The jackknife-a review , 1974 .

[217]  Michael A. Crane,et al.  Simulating Stable Stochastic Systems: III. Regenerative Processes and Discrete-Event Simulations , 1975, Oper. Res..

[218]  Paul Bratley,et al.  A guide to simulation , 1983 .

[219]  John M. Charnes,et al.  A comparison of confidence region estimators for multivariate simulation output , 1988, WSC '88.

[220]  Michael A. Crane,et al.  Simulating Stable Stochastic Systems, IV: Approximation Techniques , 1975 .

[221]  G. S. Shedler,et al.  Regeneration and Networks of Queues , 1986 .

[222]  Jean Walrand,et al.  Quick simulation of excessive backlogs in networks of queues , 1986, 1986 25th IEEE Conference on Decision and Control.

[223]  Averill M. Law,et al.  Conservation equations and variance reduction in queueing simulations , 1977, WSC '77.

[224]  George Vassilacopoulos Testing for initialization bias in simulation output , 1989, Simul..

[225]  Bernard P. Zeigler,et al.  Hierarchical, modular discrete-event modelling in an object-oriented environment , 1987 .

[226]  Michael A. Crane,et al.  Simulating Stable Stochastic Systems, I: General Multiserver Queues , 1974, JACM.

[227]  D. Iglehart Simulating stable stochastic systems, V: Comparison of ratio estimators , 1975 .

[228]  Parviz Kermani,et al.  Performance Evaluation Tool (PET): An Analysis Tool for Computer Communication Networks , 1984, IEEE J. Sel. Areas Commun..

[229]  Imrich Chlamtac,et al.  The P2 algorithm for dynamic calculation of quantiles and histograms without storing observations , 1985, CACM.