Key multiplicity issues in clinical drug development

Much progress has been made over the past decade with the development of novel methods for addressing increasingly more complex multiplicity problems arising in confirmatory Phase III clinical trials. This includes traditional problems with a single source of multiplicity, for example, analysis of multiple endpoints or dose–placebo contrasts. In addition, more advanced problems with several sources of multiplicity have attracted attention in clinical drug development. These problems include two or more families of objectives such as multiple endpoints evaluated at multiple dose levels or in multiple patient populations. This paper provides a review of concepts that play a central role in defining and solving multiplicity problems (error rate definitions) and introduces main classes of multiple testing procedures widely used in clinical trials (nonparametric, semiparametric, and parametric procedures). The paper also presents recent advances in multiplicity research, including gatekeeping procedures for clinical trials with multiple sets of objectives. The concepts and methods introduced in the paper are illustrated using several case studies on the basis of real clinical trials. Software implementation of commonly used multiple testing and gatekeeping procedures is discussed. Copyright © 2012 John Wiley & Sons, Ltd.

[1]  C. Dunnett.  A Multiple Comparison Procedure for Comparing Several Treatments with a Control , 1955 .

[2]  Z. Šidák.  Rectangular Confidence Regions for the Means of Multivariate Normal Distributions , 1967 .

[3]  K. Gabriel,et al.  On closed testing procedures with special reference to ordered analysis of variance , 1976 .

[4]  S. Holm.  A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[5]  R. Simes,et al.  An improved Bonferroni procedure for multiple tests of significance , 1986 .

[6]  B. Holland,et al.  An Improved Sequentially Rejective Bonferroni Test Procedure , 1987 .

[7]  N L Geller,et al.  The analysis of multiple endpoints in clinical trials. , 1987, Biometrics.

[8]  Jason C. Hsu,et al.  On Confidence Sets in Multiple Comparisons , 1988 .

[9]  Y. Hochberg.  A sharper Bonferroni procedure for multiple tests of significance , 1988 .

[10]  G. Hommel.  A stagewise rejective multiple test procedure based on a modified Bonferroni test , 1988 .

[11]  A. Tamhane,et al.  Step-down multiple tests for comparing treatments with a control in unbalanced one-way layouts. , 1991, Statistics in medicine.

[12]  R B D'Agostino,et al.  Multiple comparisons in over-the-counter drug clinical trials with both positive and placebo controls. , 1991, Statistics in medicine.

[13]  Charles W. Dunnett,et al.  A Step-Up Multiple Test Procedure , 1992 .

[14]  Ralph B. D'Agostino,et al.  Strategies for Dealing with Multiple Treatment Comparisons in Confirmatory Clinical Trials , 1993 .

[15]  G. E. Thomas.  Resampling‐Based Multiple Testing: Examples and Methods for p‐Value Adjustment , 1994 .

[16]  S. Sarkar,et al.  The Simes Method for Multiple Hypothesis Testing with Positively Dependent Test Statistics , 1997 .

[17]  M F Huque,et al.  Some comments on frequently used multiple endpoint adjustment methods in clinical trials. , 1997, Statistics in medicine.

[18]  R T O'Neill,et al.  Secondary endpoints cannot be validly analyzed if the primary endpoint does not demonstrate clear statistical significance. , 1997, Controlled clinical trials.

[19]  L. Hothorn,et al.  Testing strategies in multi-dose experiments including active control. , 1998, Statistics in medicine.

[20]  Multiple comparisons for an unbalanced a × b design under mixed models with interaction , 1998 .

[21]  S. Sarkar.  Some probability inequalities for ordered $\rm MTP\sb 2$ random variables: a proof of the Simes conjecture , 1998 .

[22]  M. Huque,et al.  Global Two-Group Multiple Endpoint Adjustment Methods Applied to Clinical Trials , 1999 .

[23]  R. D'Agostino,et al.  Controlling alpha in a clinical trial: the case for secondary endpoints. , 2000, Statistics in medicine.

[24]  B. Wiens.  A fixed sequence Bonferroni procedure for testing multiple endpoints , 2003 .

[25]  R. D'Agostino,et al.  Efficacy endpoint selection and multiplicity adjustment methods in clinical trials with inherent multiple endpoint issues , 2003, Statistics in medicine.

[26]  J. Heyse,et al.  Use of the false discovery rate for evaluating clinical safety data , 2004, Statistical methods in medical research.

[27]  H. Keselman,et al.  Multiple Comparison Procedures , 2005 .

[28]  B. Wiens,et al.  The Fallback Procedure for Evaluating a Single Family of Hypotheses , 2005, Journal of biopharmaceutical statistics.

[29]  R. Muirhead,et al.  Multiple Co-primary Endpoints: Medical and Statistical Solutions: A Report from the Multiple Endpoints Expert Team of the Pharmaceutical Research and Manufacturers of America , 2007 .

[30]  Sue-Jane Wang,et al.  Approaches to evaluation of treatment effect in randomized clinical trials with genomic subset , 2007, Pharmaceutical statistics.

[31]  Alex Dmitrienko,et al.  General Multistage Gatekeeping Procedures , 2008, Biometrical journal. Biometrische Zeitschrift.

[32]  Olivier Guilbaud,et al.  Simultaneous Confidence Regions Corresponding to Holm's Step‐Down Procedure and Other Closed‐Testing Procedures , 2008, Biometrical journal. Biometrische Zeitschrift.

[33]  M. Huque,et al.  A flexible fixed-sequence testing method for hierarchically ordered correlated multiple endpoints in clinical trials , 2008 .

[34]  F. Bretz,et al.  Compatible simultaneous lower confidence bounds for the Holm procedure and other Bonferroni‐based closed tests , 2008, Statistics in medicine.

[35]  T. Hothorn,et al.  Simultaneous Inference in General Parametric Models , 2008, Biometrical journal. Biometrische Zeitschrift.

[36]  A. U.S.  On weighted Hochberg procedures , 2008 .

[37]  Devan V Mehrotra,et al.  An efficient method for accommodating potentially underpowered primary endpoints , 2008, Statistics in medicine.

[38]  Mark Chang,et al.  Discussion of “Some Controversial Multiple Testing Problems in Regulatory Applications” , 2009 .

[39]  W. Brannath,et al.  A graphical approach to sequentially rejective multiple test procedures , 2009, Statistics in medicine.

[40]  Peter H. Westfall,et al.  Multiple testing methodology , 2009 .

[41]  Alex Dmitrienko,et al.  Gatekeeping Procedures in Clinical Trials , 2009 .

[42]  O. Guilbaud,et al.  A recycling framework for the construction of Bonferroni‐based multiple tests , 2009, Statistics in medicine.

[43]  Multiplicity Problems in Clinical Trials: A Regulatory Perspective , 2009 .

[44]  Mohammad F Huque,et al.  A flexible strategy for testing subgroups and overall population , 2009, Statistics in medicine.

[45]  David Wright,et al.  Discussion of “Some Controversial Multiple Testing Problems in Regulatory Applications” , 2009 .

[46]  Sue-Jane Wang,et al.  Some Controversial Multiple Testing Problems in Regulatory Applications , 2009, Journal of biopharmaceutical statistics.

[47]  J. Goeman,et al.  The Sequential Rejection Principle of Familywise Error Control , 2010, 1211.3313.

[48]  Mohammad F Huque,et al.  A consistency‐adjusted alpha‐adaptive strategy for sequential testing , 2010, Statistics in medicine.

[49]  Alex Dmitrienko,et al.  Design and Analysis Considerations in Clinical Trials With a Sensitive Subpopulation , 2010 .

[50]  Sue-Jane Wang,et al.  Challenges to multiple testing in clinical trials , 2010, Biometrical journal. Biometrische Zeitschrift.

[51]  Alex Dmitrienko,et al.  Mixtures of multiple testing procedures for gatekeeping applications in clinical trials , 2011, Statistics in medicine.

[52]  Alex Dmitrienko,et al.  Development of gatekeeping strategies in confirmatory clinical trials , 2011, Biometrical journal. Biometrische Zeitschrift.

[53]  H. Meltzer,et al.  Lurasidone in the treatment of schizophrenia: a randomized, double-blind, placebo- and olanzapine-controlled study. , 2011, The American journal of psychiatry.

[54]  Alex Dmitrienko,et al.  Chain Procedures: A Class of Flexible Closed Testing Procedures With Clinical Trial Applications , 2011 .

[55]  Alex Dmitrienko,et al.  A Mixture Gatekeeping Procedure Based on the Hommel Test for Clinical Trial Applications , 2011, Journal of biopharmaceutical statistics.

[56]  Alex Dmitrienko,et al.  Multistage and Mixture Parallel Gatekeeping Procedures in Clinical Trials , 2011, Journal of biopharmaceutical statistics.

[57]  O. Guilbaud,et al.  Confidence Regions for Bonferroni-Based Closed Tests Extended to More General Closed Tests , 2011, Journal of biopharmaceutical statistics.

[58]  Mohamed Alosh,et al.  Addressing Multiplicity Issues of a Composite Endpoint and Its Components in Clinical Trials , 2011, Journal of biopharmaceutical statistics.

[59]  Alex Dmitrienko,et al.  Three Methods for Constructing Parallel Gatekeeping Procedures in Clinical Trials , 2011, Journal of biopharmaceutical statistics.

[60]  Lei Shen,et al.  A Statistical Framework for Decision Making in Confirmatory Multipopulation Tailoring Clinical Trials , 2012 .

[61]  O. Guilbaud.  Simultaneous confidence regions for closed tests, including Holm‐, Hochberg‐, and Hommel‐related procedures , 2012, Biometrical journal. Biometrische Zeitschrift.

[62]  Alex Dmitrienko,et al.  Superchain procedures in clinical trials with multiple objectives. , 2013, Statistics in medicine.