Using elementary statistical theory, we discuss sta tistical techniques that can be used to initialize a simulation run and to determine its length, distin guishing between terminating and nonterminating sys tems and between stationary and nonstationary time series. Confidence intervals and hypothesis tests are included (see Section 2). In the case of k sys tem variants (at least 2), multiple comparison proce dures are presented which can be used to obtain simultaneously valid confidence intervals and to select a subset containing the best population, assuming a fixed number of simulation runs. Other wise ranking procedures can be used to determine the number of runs required to select the best system (Section 3). If many parameters and variables exist, selecting a limited number of combinations requires an experimental design, which must be analyzed with a regression metamodel (Section 4). The metamodel of main effects and interactions applies to both quantitative and qualitative factors; its adequacy can be tested (Subsection 4.1). Experimental design is discussed in three steps: (1) screening designs for finding the important factors, namely, 2k-P designs, random, supersaturated, and group-screening designs (Subsection 4.2); (2) augmented 2K-P designs for further exploration (Subsection 4.3); (3) response surface methodology for optimization (Subsection 4.4). Three practical variance reduction techniques are summarized, namely, control variates, antithetics, and common random numbers (Section 5). Some of the statistical methods discussed are also applicable to deterministic models of continuous systems. This paper was written for readers with only an elementary background in statistics.
[1]
F. E. Satterthwaite.
Random Balance Experimentation
,
1959
.
[2]
Martin Schatzoff,et al.
Design of Experiments in Simulator Validation
,
1975,
IBM J. Res. Dev..
[3]
K. H. Booth,et al.
Some Systematic Supersaturated Designs
,
1962
.
[4]
Robert W. Blanning,et al.
The construction and implementation of metamodels
,
1975
.
[5]
H. Robbins,et al.
A Sequential Analogue of the Behrens-Fisher Problem
,
1967
.
[6]
Jack P. C. Kleijnen,et al.
A Comment on Blanning's “Metamodel for Sensitivity Analysis: The Regression Metamodel in Simulation”
,
1975
.
[7]
George S. Fishman,et al.
Problems in the statistical analysis of simulation experiments: the comparision of means and the length of sample records
,
1967,
CACM.
[8]
George S. Fishman.
Estimation in Multiserver Queuing Simulations
,
1974,
Oper. Res..
[9]
E. J. McGrath,et al.
Application of variance reduction to large scale simulation problems
,
1974,
Comput. Oper. Res..
[10]
M. S. Patel.
GROUP-SCREENING WITH MORE THAN TWO STAGES
,
1962
.
[11]
S. Gupta.
On Some Multiple Decision (Selection and Ranking) Rules
,
1965
.
[12]
Stephen S. Lavenberg,et al.
Regenerative Simulation of a Queuing Model of an Automated Tape Library
,
1975,
IBM J. Res. Dev..
[13]
Samuel Eilon,et al.
A Simulation Study for the Design of an Air Terminal Building
,
1973,
IEEE Trans. Syst. Man Cybern..
[14]
D. Cox,et al.
Recent Work on the Design of Experiments: A Bibliography and a Review
,
1969
.