k‐FWER Control without p ‐value Adjustment, with Application to Detection of Genetic Determinants of Multiple Sclerosis in Italian Twins

Summary We show a novel approach for k‐FWER control which does not involve any correction, but only testing the hypotheses along a (possibly data‐driven) order until a suitable number of p‐values are found above the uncorrected α level. p‐values can arise from any linear model in a parametric or nonparametric setting. The approach is not only very simple and computationally undemanding, but also the data‐driven order enhances power when the sample size is small (and also when k and/or the number of tests is large). We illustrate the method on an original study about gene discovery in multiple sclerosis, in which were involved a small number of couples of twins, discordant by disease. The methods are implemented in an R package (someKfwer), freely available on CRAN.

[1]  K. Gabriel,et al.  On closed testing procedures with special reference to ordered analysis of variance , 1976 .

[2]  R L Iman,et al.  Analysis of covariance using the rank transformation. , 1982, Biometrics.

[3]  K. Fang,et al.  Generalized Multivariate Analysis , 1990 .

[4]  A. W. Kemp,et al.  Univariate Discrete Distributions , 1993 .

[5]  Yogendra P. Chaubey.  Resampling-Based Multiple Testing: Examples and Methods for p-Value Adjustment , 1993 .

[6]  S. Kropf,et al.  Multivariate tests based on left-spherically distributed linear scores , 1998 .

[7]  R. Berger,et al.  Stepwise Confidence Intervals without Multiplicity Adjustment for Dose—Response and Toxicity Studies , 1999 .

[8]  M. Salvetti,et al.  Twins: mirrors of the immune system. , 2000, Immunology today.

[9]  S. Kropf,et al.  Multiple Tests for Different Sets of Variables Using a Data‐Driven Ordering of Hypotheses, with an Application to Gene Expression Data , 2002 .

[10]  Joseph P. Romano,et al.  Generalizations of the familywise error rate , 2005, math/0507420.

[11]  S. Dudoit,et al.  Multiple Hypothesis Testing in Microarray Experiments , 2003 .

[12]  John D. Storey,et al.  Statistical significance for genomewide studies , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[13]  R. Paschke,et al.  Nonparametric multiple test procedures with data-driven order of hypotheses and with weighted hypotheses , 2004 .

[14]  M. J. van der Laan,et al.  Augmentation Procedures for Control of the Generalized Family-Wise Error Rate and Tail Probabilities for the Proportion of False Positives , 2004, Statistical applications in genetics and molecular biology.

[15]  P. Westfall,et al.  Weighted FWE-controlling methods in high-dimensional situations , 2004 .

[16]  Michael Wolf,et al.  Control of generalized error rates in multiple testing , 2007, 0710.2258.

[17]  G. Hommel,et al.  Tests for Differentiation in Gene Expression Using a Data‐Driven Order or Weights for Hypotheses , 2004, Biometrical journal. Biometrische Zeitschrift.

[18]  A. W. Kemp,et al.  Univariate Discrete Distributions: Johnson/Univariate Discrete Distributions , 2005 .

[19]  John D. Storey,et al.  Relaxed Significance Criteria for Linkage Analysis , 2006, Genetics.

[20]  Luigi Salmaso,et al.  Weighted methods controlling the multiplicity when the number of variables is much higher than the number of observations , 2006 .

[21]  Alessio Farcomeni,et al.  More Powerful Control of the False Discovery Rate Under Dependence , 2006, Stat. Methods Appl..

[22]  F Bretz,et al.  Ordered multiple comparisons with the best and their applications to dose-response studies. , 2007, Biometrics.

[23]  Roger E. Kirk,et al.  Effect magnitude: A different focus , 2007 .

[24]  A. Farcomeni.  Some Results on the Control of the False Discovery Rate under Dependence , 2007 .

[25]  Joseph P. Romano,et al.  A Generalized Sidak-Holm Procedure and Control of Generalized Error Rates under Independence , 2007, Statistical applications in genetics and molecular biology.

[26]  Alessio Farcomeni,et al.  A review of modern multiple hypothesis testing, with particular attention to the false discovery proportion , 2008, Statistical methods in medical research.

[27]  P. Rosenbaum.  Testing hypotheses in order , 2008 .

[28]  H. Cabral,et al.  Multiple Comparisons Procedures , 2008, Circulation.

[29]  Sanat K. Sarkar,et al.  Generalizing Simes' test and Hochberg's stepup procedure , 2008, 0803.1961.

[30]  P. Hall,et al.  Robustness of multiple testing procedures against dependence , 2009, 0903.0464.