An Evaluation of Ten Pairwise Multiple Comparison Procedures by Monte Carlo Methods

Abstract Computer simulation techniques were used to study the Type I and Type III error rates and the correct decision rates for ten pairwise multiple comparison procedures. Results indicated that Scheffe's test, Tukey's test, and the Student-Newman-Keuls test are less appropriate than either the least significant difference with the restriction that the analysis of variance F value be significant at α = .05, two Bayesian modifications of the least significant difference, or Duncan's multiple range test. Because of its ease of application, many researchers may prefer the restricted least significant difference.

[1]  H. Scheffé.  A METHOD FOR JUDGING ALL CONTRASTS IN THE ANALYSIS OF VARIANCE , 1953 .

[2]  D. B. Duncan.  MULTIPLE RANGE AND MULTIPLE F TESTS , 1955 .

[3]  H. Leon Harter,et al.  Error Rates and Sample Sizes for Range Tests in Multiple Comparisons , 1957 .

[4]  M. E. Muller,et al.  A Note on the Generation of Random Normal Deviates , 1958 .

[5]  H. Leon Harter,et al.  THE PROBABILITY INTEGRALS OF THE RANGE AND OF THE STUDENTIZED RANGE. PROBABILITY INTEGRAL AND PERCENTAGE POINTS OF THE STUDENTIZED RANGE; CRITICAL VALUES FOR DUNCAN'S NEW MULTIPLE RANGE TEST , 1959 .

[6]  R. G. Steel.  163 Query: Error Rates in Multiple Comparisons , 1961 .

[7]  L. N. Balaam.  MULTIPLE COMPARISONS—A SAMPLING EXPERIMENT , 1963 .

[8]  D. B. Duncan,et al.  A Bayesian Approach to Multiple Comparisons , 1965 .

[9]  Kenneth D. Hopkins,et al.  Comments: A Schema for Proper Utilization of Multiple Comparisons in Research and a Case Study1 , 1967 .

[10]  George Marsaglia,et al.  One-line random number generators and their use in combinations , 1968, Commun. ACM.

[11]  Comments: Multiple Comparisons in Research: A Response to a Comment , 1969 .

[12]  Multiple Comparisons in Research: A Response to a Comment , 1969 .

[13]  R. A. Waller,et al.  A Bayes Rule for the Symmetric Multiple Comparisons Problem , 1969 .

[14]  C. Dunnett.  272. Query: Multiple Comparison Tests , 1970 .

[15]  S. G. Carmer,et al.  Detection of Differences Between Means: A Monte Carlo Study of Five Pairwise Multiple Comparison Procedures1 , 1971 .

[16]  T. Boardman,et al.  Graphical Monte Carlo type I error rates for multiple comparison procedures. , 1971, Biometrics.