On stepdown control of the false discovery proportion

Consider the problem of testing multiple null hypotheses. A clas- sical approach to dealing with the multiplicity problem is to restrict attention to procedures that control the familywise error rate (FWER), the probabil- ity of even one false rejection. However, if s is large, control of the FWER is so stringent that the ability of a procedure which controls the FWER to detect false null hypotheses is limited. Consequently, it is desirable to consider other measures of error control. We will consider methods based on control of the false discovery proportion (FDP) defined by the number of false rejec- tions divided by the total number of rejections (defined to be 0 if there are no rejections). The false discovery rate proposed by Benjamini and Hochberg (1995) controls E(FDP). Here, we construct methods such that, for any and �, P {FDP > } ≤ �. Based on p-values of individual tests, we consider stepdown procedures that control the FDP, without imposing dependence as- sumptions on the joint distribution of the p-values. A greatly improved version of a method given in Lehmann and Romano (10) is derived and generalized to provide a means by which any sequence of nondecreasing constants can be rescaled to ensure control of the FDP. We also provide a stepdown procedure that controls the FDR under a dependence assumption.

[1]  S. Sarkar,et al.  The Simes Method for Multiple Hypothesis Testing with Positively Dependent Test Statistics , 1997 .

[2]  G. Hommel.  Tests of the overall hypothesis for arbitrary dependence structures , 1983 .

[3]  R. Simes,et al.  An improved Bonferroni procedure for multiple tests of significance , 1986 .

[4]  H. Keselman,et al.  Multiple Comparison Procedures , 2005 .

[5]  Larry Wasserman,et al.  False Discovery Rates for Random Fields , 2003 .

[6]  S. Holm.  A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[7]  Y. Benjamini,et al.  A step-down multiple hypotheses testing procedure that controls the false discovery rate under independence , 1999 .

[8]  M. J. van der Laan,et al.  Augmentation Procedures for Control of the Generalized Family-Wise Error Rate and Tail Probabilities for the Proportion of False Positives , 2004, Statistical applications in genetics and molecular biology.

[9]  L. Wasserman,et al.  A stochastic process approach to false discovery control , 2004, math/0406519.

[10]  Joseph P. Romano,et al.  Generalizations of the familywise error rate , 2005, math/0507420.

[11]  A. Tamhane,et al.  Multiple Comparison Procedures , 1989 .

[12]  Joseph P. Romano,et al.  Stepup procedures for control of generalizations of the familywise error rate , 2006, math/0611266.

[13]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[14]  Y. Benjamini,et al.  THE CONTROL OF THE FALSE DISCOVERY RATE IN MULTIPLE TESTING UNDER DEPENDENCY , 2001 .

[15]  S. Sarkar.  Some Results on False Discovery Rate in Stepwise multiple testing procedures , 2002 .

[16]  R. Simon,et al.  Controlling the number of false discoveries: application to high-dimensional genomic data , 2004 .

[17]  S. Sarkar.  Some probability inequalities for ordered $\rm MTP\sb 2$ random variables: a proof of the Simes conjecture , 1998 .