Multiple Comparison Procedures

Multiple comparison procedures (MCPs) are frequently adopted by applied researchers to locate specific differences between treatment groups. That is, omnibus test statistics, such as the analysis of variance F test, can only signify that effects are present, not which specific groups differ from one another (when there are more than two groups). In our paper, we discuss MCPs that can be used to investigate simple pairwise differences between treatment group means, as well as MCPs that can be used to examine complex comparisons (i.e., nonpairwise comparisons) between treatment group means. In particular, we discuss simultaneous as well as stepwise MCPs, emphasizing procedures that can be utilized when the derivational assumptions of the classical procedures of normality and variance homogeneity do not hold. Keywords: pairwise and complex comparisons; simultaneous and stepwise procedures; Type I error rates; robust procedures

[1]  Rand R. Wilcox,et al.  Comparing the means of two independent groups , 2007 .

[2]  H J Keselman,et al.  A generally robust approach to hypothesis testing in independent and correlated groups designs. , 2003, Psychophysiology.

[3]  S. Dudoit,et al.  Multiple Hypothesis Testing in Microarray Experiments , 2003 .

[4]  Siu Hung Cheung,et al.  Familywise robustness criteria for multiple‐comparison procedures , 2002 .

[5]  Y. Benjamini,et al.  On the Adaptive Control of the False Discovery Rate in Multiple Testing With Independent Statistics , 2000 .

[6]  Robert A. Cribbie,et al.  The pairwise multiple comparison multiplicity problem: An alternative approach to familywise and comparison wise Type I error control. , 1999 .

[7]  Carl J. Huberty,et al.  Statistical Practices of Educational Researchers: An Analysis of their ANOVA, MANOVA, and ANCOVA Analyses , 1998 .

[8]  H. Keselman.  Testing treatment effects in repeated measures designs: an update for psychophysiological researchers. , 1998, Psychophysiology.

[9]  J. Kromrey,et al.  Power and Type I Error Rates of New Pairwise Multiple Comparison Procedures Under Heterogeneous Variances , 1995 .

[10]  C. Dayton,et al.  Detecting patterns of bivariate mean vectors using model‐selection criteria , 1995 .

[11]  H. Keselman,et al.  Approximate degrees of freedom tests: A unified perspective on testing for mean equality. , 1995 .

[12]  Rand R. Wilcox,et al.  ANOVA: The practical importance of heteroscedastic methods, using trimmed means versus means, and designing simulation studies , 1995 .

[13]  H. Keselman,et al.  Improved Repeated Measures Stepwise Multiple Comparison Procedures , 1995 .

[14]  Joel R. Levin,et al.  New developments in pairwise multiple comparisons : some powerful and practicable procedures , 1991 .

[15]  D. Rom.  A sequentially rejective test procedure based on a modified Bonferroni inequality , 1990 .

[16]  S. Sheather,et al.  Robust Estimation & Testing: Staudte/Robust , 1990 .

[17]  K J Rothman,et al.  No Adjustments Are Needed for Multiple Comparisons , 1990, Epidemiology.

[18]  Y. Hochberg.  A sharper Bonferroni procedure for multiple tests of significance , 1988 .

[19]  R. Wilcox.  A new alternative to the ANOVA F and new results on James's second-order method , 1988 .

[20]  B. Holland,et al.  An Improved Sequentially Rejective Bonferroni Test Procedure , 1987 .

[21]  A. Hayter.  The Maximum Familywise Error Rate of Fisher's Least Significant Difference Test , 1986 .

[22]  J. Shaffer.  Modified Sequentially Rejective Multiple Test Procedures , 1986 .

[23]  C. Dunnett.  Pairwise Multiple Comparisons in the Unequal Variance Case , 1980 .

[24]  R. Welsch.  Stepwise Multiple Comparison Procedures , 1977 .

[25]  A. M. Gross.  Confidence Interval Robustness with Long-Tailed Symmetric Distributions , 1976 .

[26]  Morton B. Brown,et al.  372: The Anova and Multiple Comparisons for Data with Heterogeneous Variances , 1974 .

[27]  H. Akaike.  A new look at the statistical model identification , 1974 .

[28]  W. J. Dixon,et al.  The approximate behaviour and performance of the two-sample trimmed t , 1973 .

[29]  Paul A. Games,et al.  Multiple Comparisons of Means , 1971 .

[30]  W. Wilson,et al.  A note on the incosistency inherent in the necessity to perform multiple comparisons. , 1962, Psychological bulletin.

[31]  T. A. Ryan,et al.  The experiment as the unit for computing rates of error. , 1962, Psychological bulletin.

[32]  O. J. Dunn.  Multiple Comparisons among Means , 1961 .

[33]  T. A. Ryan,et al.  Significance tests for multiple comparison of proportions, variances, and other statistics. , 1960, Psychological bulletin.

[34]  Clyde Young Kramer,et al.  Extension of multiple range tests to group means with unequal numbers of replications , 1956 .

[35]  D. B. Duncan.  MULTIPLE RANGE AND MULTIPLE F TESTS , 1955 .

[36]  M. Keuls,et al.  The use of the „studentized range” in connection with an analysis of variance , 1952, Euphytica.

[37]  B. L. Welch.  ON THE COMPARISON OF SEVERAL MEAN VALUES: AN ALTERNATIVE APPROACH , 1951 .

[38]  Satterthwaite Fe.  An approximate distribution of estimates of variance components. , 1946 .

[39]  D. Newman,et al.  THE DISTRIBUTION OF RANGE IN SAMPLES FROM A NORMAL POPULATION, EXPRESSED IN TERMS OF AN INDEPENDENT ESTIMATE OF STANDARD DEVIATION , 1939 .

[40]  E. S. Pearson.  THE ANALYSIS OF VARIANCE IN CASES OF NON-NORMAL VARIATION , 1931 .

[41]  R. Wilcox.  Three Multiple Comparison Procedures for Trimmed Means , 1995 .

[42]  S. Olejnik,et al.  Power of Pairwise Multiple Comparisons in the Unequal Variance Case. , 1994 .

[43]  Joel R. Levin,et al.  A controlled, powerful multiple-comparison strategy for several situations. , 1994 .

[44]  H. Keselman,et al.  Maximum familywise Type I error rate: The least significant difference, Newman-Keuls, and other multiple comparison procedures. , 1991 .

[45]  T. Micceri.  The unicorn, the normal curve, and other improbable creatures. , 1989 .

[46]  Rand R. Wilcox,et al.  New monte carlo results on the robustness of the anova f, w and f statistics , 1986 .

[47]  A. Hayter.  A Proof of the Conjecture that the TUKEY-KRAMER Multiple Comparisons Procedure is Conservative , 1984 .

[48]  Curtis Hardyck,et al.  Error rates for multiple comparison methods: Some evidence concerning the frequency of erroneous conclusions. , 1969 .

[49]  T. A. Ryan.  Multiple comparison in psychological research. , 1959 .