Multiplicity Issues in Microarray Experiments

OBJECTIVES Discussion of different error concepts relevant to microarray experiments. Review of some commonly used multiple testing procedures. Comparison of different approaches as applied to gene expression data. METHODS This article focuses on familywise error rate (FWER) and false discovery rate (FDR) controlling procedures. Methods under investigation include: Bonferroni-type methods and their improvements (including resampling approaches), modified Bonferroni methods, data-driven approaches, as well as the linear step-up method and its modifications. Particular emphasis lies on the description of the assumptions, advantages and limitations for the investigated methods. RESULTS FWER controlling procedures are often too conservative in high dimensional screening studies. A better balance between the raw P-values and the stringent FWER-adjusted P-values may be required in many situations, as provided by FDR controlling and related procedures. CONCLUSIONS The questions remain open, which error concept to apply and which multiple testing procedure to use. Although we believe that the FDR or one of its variants will be applied more often in the future, longterm experience with microarray technology is missing and thus the validity of appropriate multiple test procedures cannot yet be assessed for microarray data analysis.

[1]  J. Booth,et al.  Resampling-Based Multiple Testing. , 1994 .

[2]  R. Simon,et al.  Controlling the number of false discoveries: application to high-dimensional genomic data , 2004 .

[3]  H. Finner,et al.  On the False Discovery Rate and Expected Type I Errors , 2001 .

[4]  J. Mesirov,et al.  Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. , 1999, Science.

[5]  Huey-miin Hsueh,et al.  Comparison of Methods for Estimating the Number of True Null Hypotheses in Multiplicity Testing , 2003, Journal of biopharmaceutical statistics.

[6]  Charles W. Dunnett,et al.  Power and sample size comparisons of stepwise FWE and FDR controlling test procedures in the normal many-one case , 2004 .

[7]  Tao Wang,et al.  Screening for Differential Gene Expressions from Microarray Data , 2006 .

[8]  R. Simes,et al.  An improved Bonferroni procedure for multiple tests of significance , 1986 .

[9]  Yongchao Ge.  Resampling-based Multiple Testing for Microarray Data Analysis , 2003 .

[10]  John D. Storey,et al.  Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: a unified approach , 2004 .

[11]  S. Dudoit,et al.  Multiple Hypothesis Testing in Microarray Experiments , 2003 .

[12]  Sandrine Dudoit,et al.  Multiple Testing. Part I. Single-Step Procedures for Control of General Type I Error Rates , 2004, Statistical applications in genetics and molecular biology.

[13]  G. Hommel.  A stagewise rejective multiple test procedure based on a modified Bonferroni test , 1988 .

[14]  P. Westfall,et al.  Optimally weighted, fixed sequence and gatekeeper multiple testing procedures , 2001 .

[15]  J. Shaffer.  Modified Sequentially Rejective Multiple Test Procedures , 1986 .

[16]  J. Weller,et al.  A new approach to the problem of multiple comparisons in the genetic dissection of complex traits. , 1998, Genetics.

[17]  M. J. van der Laan,et al.  Augmentation Procedures for Control of the Generalized Family-Wise Error Rate and Tail Probabilities for the Proportion of False Positives , 2004, Statistical applications in genetics and molecular biology.

[18]  John D. Storey,et al.  Empirical Bayes Analysis of a Microarray Experiment , 2001 .

[19]  John D. Storey.  A direct approach to false discovery rates , 2002 .

[20]  Y. Benjamini,et al.  Resampling-based false discovery rate controlling multiple test procedures for correlated test statistics , 1999 .

[21]  P. Seeger.  A Note on a Method for the Analysis of Significances en masse , 1968 .

[22]  S. Sarkar,et al.  The Simes Method for Multiple Hypothesis Testing with Positively Dependent Test Statistics , 1997 .

[23]  R. Tibshirani,et al.  Significance analysis of microarrays applied to the ionizing radiation response , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[24]  P. N. Somerville.  FDR step-down and step-up procedures for the correlated case , 2004 .

[25]  Y. Benjamini,et al.  A step-down multiple hypotheses testing procedure that controls the false discovery rate under independence , 1999 .

[26]  G. Hommel,et al.  Tests for Differentiation in Gene Expression Using a Data‐Driven Order or Weights for Hypotheses , 2004, Biometrical journal. Biometrische Zeitschrift.

[27]  H. Keselman,et al.  Multiple Comparison Procedures , 2005 .

[28]  J. Troendle,et al.  Stepwise normal theory multiple test procedures controlling the false discovery rate , 2000 .

[29]  Sandrine Dudoit,et al.  Multiple Testing. Part II. Step-Down Procedures for Control of the Family-Wise Error Rate , 2004, Statistical applications in genetics and molecular biology.

[30]  Y. Hochberg.  A sharper Bonferroni procedure for multiple tests of significance , 1988 .

[31]  D. Zaykin,et al.  Using the false discovery rate approach in the genetic dissection of complex traits: a response to Weller et al. , 2000, Genetics.

[32]  S. Kropf,et al.  Multivariate tests based on left-spherically distributed linear scores , 1998 .

[33]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .