On Using the Bootstrap for Multiple Comparisons

There are many ways to bootstrap data for multiple comparisons procedures. Methods described here include (i) bootstrap (parametric and nonparametric) as a generalization of classical normal-based MaxT methods, (ii) bootstrap as an approximation to exact permutation methods, (iii) bootstrap as a generator of realistic null data sets, and (iv) bootstrap as a generator of realistic non-null data sets. Resampling of MinP versus MaxT is discussed, and the use of the bootstrap for closed testing is also presented. Applications to biopharmaceutical statistics are given.

[1]  K. Gabriel,et al.  On closed testing procedures with special reference to ordered analysis of variance , 1976 .

[2]  James F Troendle,et al.  Multiple Testing with Minimal Assumptions , 2008, Biometrical journal. Biometrische Zeitschrift.

[3]  E. Korn,et al.  An Example of Slow Convergence of the Bootstrap in High Dimensions , 2004 .

[4]  R. Beran,et al.  Bootstrap Tests and Confidence Regions for Functions of a Covariance Matrix , 1985 .

[5]  Julian L. Simon,et al.  Resampling: A Tool for Everyday Statistical Work , 1991 .

[6]  David Hinkley,et al.  Bootstrap Methods: Another Look at the Jackknife , 2008 .

[7]  Discussion of 'False Discovery Rate Adjusted Multiple Confidence Intervals for Selected Parameters' by Y. Benjamini and D. Yekutieli , 2005 .

[8]  L. Hothorn,et al.  A Dunnett-Type Procedure for Multiple Endpoints , 2011 .

[9]  S. Young,et al.  p Value Adjustments for Multiple Tests in Multivariate Binomial Models , 1989 .

[10]  P. Westfall,et al.  Multiple Testing of General Contrasts , 2007 .

[11]  Pairwise comparisons of the means of skewed data , 2000 .

[12]  P. Westfall,et al.  Multiple comparisons and multiple tests using SAS , 2011 .

[13]  Rudolf Beran,et al.  Balanced Simultaneous Confidence Sets , 1988 .

[14]  C. Dunnett.  A Multiple Comparison Procedure for Comparing Several Treatments with a Control , 1955 .

[15]  P. Westfall,et al.  Multiple McNemar Tests , 2010, Biometrics.

[16]  L. Hothorn,et al.  Comparison of Exact and Resampling Based Multiple Testing Procedures , 2003 .

[17]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[18]  S. Dudoit,et al.  Multiple Hypothesis Testing in Microarray Experiments , 2003 .

[19]  P. Westfall.  Improving Power by Dichotomizing (Even Under Normality) , 2011 .

[20]  S. S. Young,et al.  Resampling-Based Multiple Testing: Examples and Methods for p-Value Adjustment , 1993 .

[21]  S. Holm.  A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[22]  J. Booth,et al.  Resampling-Based Multiple Testing. , 1994 .

[23]  Frank Bretz,et al.  On Multiple Comparisons in R ∗ , 2004 .

[24]  P. Westfall,et al.  Gatekeeping strategies for clinical trials that do not require all primary effects to be significant , 2003, Statistics in medicine.

[25]  P. Westfall.  Simultaneous small-sample multivariate Bernoulli confidence intervals. , 1985, Biometrics.

[26]  C. F. Wu.  JACKKNIFE , BOOTSTRAP AND OTHER RESAMPLING METHODS IN REGRESSION ANALYSIS ' BY , 2008 .