Closed procedures are better and often admit a shortcut

Abstract It is ‘common knowledge’ that closed multiple test procedures are typically better than others in terms of power. In the first part of this paper we prove that closed procedures constitute a complete class in the class of all coherent procedures, and, also, that α-exhaustive procedures constitute a minimal complete class there. However, closed procedures are not always ‘monotone in p-values’ and are often computationally inefficient. Thus, in the second part of this paper we first investigate conditions for a closed procedure to be ‘monotone in p-values’, and second, we shed further light on the possibility of reducing a given closed procedure to a step-down sequentially rejective procedure. We give a sufficient condition for a closure based on some generalized Simes (1986) , (Biometrika 73, 751–754) type tests to be ‘monotone in p-values’ and clarify Hochberg and Tamhane (1987) , (Multiple Comparison Procedures, Wiley, New York) general result on the existence of a shortcut in families satisfying the free combination condition. We comment on the interrelationship between ‘stepwise’ (step-down and step-up), ‘monotone in p-values’, and ‘sequentially rejective’ procedures, in families satisfying the ‘free combinations condition’ as well as in families involving ‘logically related’ hypotheses.

[1]  Wei Liu,et al.  Multiple Tests of a Non‐Hierarchical Finite Family of Hypotheses , 1996 .

[2]  Gerhard Hommel,et al.  Multiple Hypotheses Testing , 1993 .

[3]  R. Simes,et al.  An improved Bonferroni procedure for multiple tests of significance , 1986 .

[4]  G. Hommel.  A stagewise rejective multiple test procedure based on a modified Bonferroni test , 1988 .

[5]  Generalized maximum range tests for pairwise comparisons of several populations , 1989 .

[6]  G. Hommel.  Multiple test procedures for arbitrary dependence structures , 1986 .

[7]  C. Dunnett.  A Multiple Comparison Procedure for Comparing Several Treatments with a Control , 1955 .

[8]  B. Holland,et al.  A new closed multiple testing procedure for hierarchical families of hypotheses , 1995 .

[9]  A. Tamhane,et al.  Multiple Comparison Procedures , 1989 .

[10]  E. Lehmann,et al.  On a Fundamental Theorem in Multiple Comparisons , 1977 .

[11]  S. Holm.  A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[12]  K. Ruben Gabriel,et al.  Closure of the Newman-Keuls Multiple Comparisons Procedure , 1981 .

[13]  Y. Hochberg.  A sharper Bonferroni procedure for multiple tests of significance , 1988 .

[14]  Yosef Hochberg,et al.  Extensions of multiple testing procedures based on Simes' test , 1995 .

[15]  Umesh D. Naik,et al.  Some selection rules for comparing p processes with a standard , 1975 .

[16]  D. Rom.  A sequentially rejective test procedure based on a modified Bonferroni inequality , 1990 .

[17]  E. Sonnemann,et al.  Vollständigkeitssätze für Multiple Testprobleme , 1988 .

[18]  J. Shaffer.  Modified Sequentially Rejective Multiple Test Procedures , 1986 .

[19]  Michele Conforti,et al.  Sequentially rejective pairwise testing procedures , 1987 .

[20]  K. Gabriel,et al.  On closed testing procedures with special reference to ordered analysis of variance , 1976 .