α‐investing: a procedure for sequential control of expected false discoveries

Summary.  α‐investing is an adaptive sequential methodology that encompasses a large family of procedures for testing multiple hypotheses. All control mFDR, which is the ratio of the expected number of false rejections to the expected number of rejections. mFDR is a weaker criterion than the false discovery rate, which is the expected value of the ratio. We compensate for this weakness by showing that α‐investing controls mFDR at every rejected hypothesis. α‐investing resembles α‐spending that is used in sequential trials, but it has a key difference. When a test rejects a null hypothesis, α‐investing earns additional probability towards subsequent tests. α‐investing hence allows us to incorporate domain knowledge into the testing procedure and to improve the power of the tests. In this way, α‐investing enables the statistician to design a testing procedure for a specific problem while guaranteeing control of mFDR.

[1]  S. Holm A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[2]  J. Rissanen A UNIVERSAL PRIOR FOR INTEGERS AND ESTIMATION BY MINIMUM DESCRIPTION LENGTH , 1983 .

[3]  R. Simes,et al.  An improved Bonferroni procedure for multiple tests of significance , 1986 .

[4]  J. Tukey The Philosophy of Multiple Comparisons , 1991 .

[5]  William J. Thompson,et al.  The collected works of john w. tukey , 1991 .

[6]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[7]  J F Troendle,et al.  A permutational step-up method of testing multiple outcomes. , 1996, Biometrics.

[8]  S. Sarkar Some probability inequalities for ordered $\rm MTP\sb 2$ random variables: a proof of the Simes conjecture , 1998 .

[9]  W. Lehmacher,et al.  Adaptive Sample Size Calculations in Group Sequential Trials , 1999, Biometrics.

[10]  Y. Benjamini,et al.  THE CONTROL OF THE FALSE DISCOVERY RATE IN MULTIPLE TESTING UNDER DEPENDENCY , 2001 .

[11]  H. Finner,et al.  On the False Discovery Rate and Expected Type I Errors , 2001 .

[12]  John D. Storey A direct approach to false discovery rates , 2002 .

[13]  L. Wasserman,et al.  Operating characteristics and extensions of the false discovery rate procedure , 2002 .

[14]  Dean P. Foster,et al.  Universal codes for finite sequences of integers drawn from a monotone distribution , 2002, IEEE Trans. Inf. Theory.

[15]  S. Dudoit,et al.  Multiple Hypothesis Testing in Microarray Experiments , 2003 .

[16]  John D. Storey The positive false discovery rate: a Bayesian interpretation and the q-value , 2003 .

[17]  A. Tsiatis,et al.  On the inefficiency of the adaptive design for monitoring clinical trials , 2003 .

[18]  Chen-An Tsai,et al.  Estimation of False Discovery Rates in Multiple Testing: Application to Gene Microarray Data , 2003, Biometrics.

[19]  L. Wasserman,et al.  A stochastic process approach to false discovery control , 2004, math/0406519.

[20]  Bradley Efron Stanford Selection and Estimation for Large-Scale Simultaneous Inference , 2004 .

[21]  Peter Bühlmann,et al.  Lower bounds for the number of false null hypotheses for multiple testing of associations under general dependence structures , 2005 .

[22]  L. Wasserman,et al.  False discovery control with p-value weighting , 2006 .

[23]  N. Meinshausen,et al.  Estimating the proportion of false null hypotheses among a large number of independently tested hypotheses , 2005, math/0501289.

[24]  B. Efron Size, power and false discovery rates , 2007, 0710.2245.