The Use of Multiple Ranking Procedures to Analyze Simulations of Management Systems: A Tutorial

This paper describes the use of multiple ranking procedures to analyze data generated from computer simulation experiments with models of management systems. After outlining the rationale for the use of multiple ranking procedures with computer simulation experiments and defining some basic terminology, we examine several specific multiple ranking procedures. Careful attention is given to the assumptions underlying the different multiple ranking procedures and the extent to which these assumptions are satisfied by the data generated by simulation experiments. An example model is included to illustrate the applicability of multiple ranking procedures to simulation experiments in management science.

[1]  Robert E. Bechhofer,et al.  A Sequential Multiple-Decision Procedure for Selecting the Best One of Several Normal Populations with a Common Unknown Variance, and Its Use with Various Experimental Designs , 1958 .

[2]  Frederick Mosteller,et al.  A $k$-Sample Slippage Test for an Extreme Population , 1948 .

[3]  R. R. Bahadur.  On a Problem in the Theory of k Populations , 1950 .

[4]  H. Scheffé.  The Analysis of Variance , 1960 .

[5]  R. Bechhofer.  A Single-Sample Multiple Decision Procedure for Ranking Means of Normal Populations with known Variances , 1954 .

[6]  Robert E. Bechhofer,et al.  Sequential Identification and Ranking Procedures. , 1968 .

[7]  Thomas H. Wonnacott,et al.  Methods for analyzing data from computer simulation experiments , 1967, CACM.

[8]  Akio Kudô,et al.  On the invariant multiple decision procedures , 1956 .

[9]  Milton Sobel,et al.  Selecting the best one of several binomial populations , 1957 .

[10]  Shanti S. Gupta,et al.  On the smallest of several correlated F statistics , 1962 .

[11]  Shanti S. Gupta,et al.  On a Statistic which Arises in Selection and Ranking Problems , 1957 .

[12]  C. Dunnett,et al.  Approximations to the probability integral and certain percentage points of a multivariate analogue of Student's t-distribution* , 1955 .

[13]  H. Robbins,et al.  A Sequential Procedure for Selecting the Largest of $k$ Means , 1968 .

[14]  C. Dunnett.  On Selecting the Largest of k Normal Population Means , 1960 .

[15]  Shanti S. Gupta,et al.  On selecting a subset containing the population with the smallest variance , 1962 .

[16]  Alan G. Merten,et al.  Computer simulation experiments with models of economic systems , 1971 .

[17]  F. Mosteller,et al.  Significance Levels for a k-Sample Slippage Test , 1950 .

[18]  Th. H. Naylor,et al.  Some Methods for Evaluating the Effects of Economic Policies Using Simulation Experiments , 1968 .

[19]  Michie Shirafuji,et al.  Note on the Determination of the Replication Number for the Slippage Problem in r-way Layout , 1956 .

[20]  V. J. Bofinger.  THE k‐SAMPLE SLIPPAGE PROBLEM1 , 1965 .

[21]  Johann Pfanzagl,et al.  Ein kombiniertes test & klassifikations-problem , 1959 .

[22]  R. Bechhofer,et al.  A Single-Sample Multiple Decision Procedure for Ranking Variances of Normal Populations , 1954 .

[23]  E. Paulson.  A Sequential Procedure for Selecting the Population with the Largest Mean from $k$ Normal Populations , 1964 .

[24]  Kiron Chandra Seal,et al.  On a class of decision procedures for ranking means , 1955 .

[25]  Thomas H. Naylor,et al.  The application of sequential sampling to simulation , 1970, Commun. ACM.

[26]  E. L. Lehmann,et al.  SOME MODEL I PROBLEMS OF SELECTION , 1961 .

[27]  Robert E. Bechhofer,et al.  A SEQUENTIAL MULTIPLE-DECISION PROCEDURE FOR SELECTING THE BEST ONE OF SEVERAL NORMAL POPULATIONS WITH A COMMON UNKNOWN VARIANCE. II. MONTE CARLO SAMPLING RESULTS AND NEW COMPUTING FORMULAE , 1962 .

[28]  C. Dunnett,et al.  A TOW-SAMPLE MULTIPLE DECISION PROCEDURE FOR RANKING MEANS OF NORMAL POPULATIONS WITH A COMMON UNKNOWN VARIANCE , 1954 .

[29]  E. Paulson,et al.  Sequential Estimation and Closed Sequential Decision Procedures , 1964 .

[30]  M. L. Chambers,et al.  Use of double sampling for selecting best population , 1964 .

[31]  E. L. Lehmann,et al.  A Theory of Some Multiple Decision Problems. II , 1957 .

[32]  C. Dunnett.  A Multiple Comparison Procedure for Comparing Several Treatments with a Control , 1955 .

[33]  E. Paulson,et al.  An Optimum Solution to the $k$-Sample Slippage Problem for the Normal Distribution , 1952 .

[34]  Thomas H. Naylor,et al.  A Dynamic Model of the Firm , 1965 .

[35]  S. Gupta.  On Some Multiple Decision (Selection and Ranking) Rules , 1965 .

[36]  Shanti S. Gupta,et al.  On Selecting a Subset Which Contains All Populations Better Than a Standard , 1958 .

[37]  C. Dunnett,et al.  A BIVARIATE GENERALIZATION OF STUDENT'S t-DISTRIBUTION, WITH TABLES FOR CERTAIN SPECIAL CASES , 1954 .

[38]  E. Paulson,et al.  A Sequential Procedure for Comparing Several Experimental Categories with a Standard or Control , 1962 .

[39]  P. N. Somerville.  Some problems of optimum sampling , 1954 .

[40]  J. Ramberg,et al.  A Comparison of the Performance Characteristics of Two Sequential Procedures for Ranking Means of Normal Populations , 1966 .

[41]  E. Paulson.  On the Comparison of Several Experimental Categories with a Control , 1952 .