A graphical approach to sequentially rejective multiple test procedures

For clinical trials with multiple treatment arms or endpoints a variety of sequentially rejective, weighted Bonferroni‐type tests have been proposed, such as gatekeeping procedures, fixed sequence tests, and fallback procedures. They allow to map the difference in importance as well as the relationship between the various research questions onto an adequate multiple test procedure. Since these procedures rely on the closed test principle, they usually require the explicit specification of a large number of intersection hypotheses tests. The underlying test strategy may therefore be difficult to communicate. We propose a simple iterative graphical approach to construct and perform such Bonferroni‐type tests. The resulting multiple test procedures are represented by directed, weighted graphs, where each node corresponds to an elementary hypothesis, together with a simple algorithm to generate such graphs while sequentially testing the individual hypotheses. The approach is illustrated with the visualization of several common gatekeeping strategies. A case study is used to illustrate how the methods from this article can be used to tailor a multiple test procedure to given study objectives. Copyright © 2008 John Wiley & Sons, Ltd.

[1]  K. Gabriel,et al.  On closed testing procedures with special reference to ordered analysis of variance , 1976 .

[2]  S. Holm A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[3]  Michele Conforti,et al.  Sequentially rejective pairwise testing procedures , 1987 .

[5]  S. S. Young,et al.  Resampling-Based Multiple Testing: Examples and Methods for p-Value Adjustment , 1993 .

[6]  L. Hothorn,et al.  Testing strategies in multi-dose experiments including active control. , 1998, Statistics in medicine.

[7]  P. Westfall,et al.  Optimally weighted, fixed sequence and gatekeeper multiple testing procedures , 2001 .

[8]  Martin Posch,et al.  Multiple Testing for Identifying Effective and Safe Treatments , 2001 .

[9]  B. Wiens A fixed sequence Bonferroni procedure for testing multiple endpoints , 2003 .

[10]  P. Westfall,et al.  Gatekeeping strategies for clinical trials that do not require all primary effects to be significant , 2003, Statistics in medicine.

[11]  Xun Chen,et al.  The application of enhanced parallel gatekeeping strategies , 2005, Statistics in medicine.

[12]  B. Wiens,et al.  The Fallback Procedure for Evaluating a Single Family of Hypotheses , 2005, Journal of biopharmaceutical statistics.

[13]  D. Edwards,et al.  Constructing multiple test procedures for partially ordered hypothesis sets , 2007, Statistics in medicine.

[14]  Olivier Guilbaud,et al.  Bonferroni Parallel Gatekeeping – Transparent Generalizations, Adjusted P ‐values, and Short Direct Proofs , 2007, Biometrical journal. Biometrische Zeitschrift.

[15]  G. Hommel,et al.  Powerful short‐cuts for multiple testing procedures with special reference to gatekeeping strategies , 2007, Statistics in medicine.

[16]  Frank Bretz,et al.  Aesthetics and Power Considerations in Multiple Testing – A Contradiction? , 2008, Biometrical journal. Biometrische Zeitschrift.

[17]  Olivier Guilbaud,et al.  Simultaneous Confidence Regions Corresponding to Holm's Step‐Down Procedure and Other Closed‐Testing Procedures , 2008, Biometrical journal. Biometrische Zeitschrift.

[18]  F. Bretz,et al.  Compatible simultaneous lower confidence bounds for the Holm procedure and other Bonferroni‐based closed tests , 2008, Statistics in medicine.