Increasing efficiency of preclinical research by group sequential designs

Despite the potential benefits of sequential designs, studies evaluating treatments or experimental manipulations in preclinical experimental biomedicine almost exclusively use classical block designs. Our aim with this article is to bring the existing methodology of group sequential designs to the attention of researchers in the preclinical field and to clearly illustrate its potential utility. Group sequential designs can offer higher efficiency than traditional methods and are increasingly used in clinical trials. Using simulation of data, we demonstrate that group sequential designs have the potential to improve the efficiency of experimental studies, even when sample sizes are very small, as is currently prevalent in preclinical experimental biomedicine. When simulating data with a large effect size of d = 1 and a sample size of n = 18 per group, sequential frequentist analysis consumes in the long run only around 80% of the planned number of experimental units. In larger trials (n = 36 per group), additional stopping rules for futility lead to the saving of resources of up to 30% compared to block designs. We argue that these savings should be invested to increase sample sizes and hence power, since the currently underpowered experiments in preclinical biomedicine are a major threat to the value and predictiveness in this research domain.

[1]  Felix D. Schönbrodt,et al.  Sequential Hypothesis Testing With Bayes Factors: Efficiently Testing Mean Differences , 2017, Psychological methods.

[2]  Deepak L. Bhatt,et al.  Adaptive Designs for Clinical Trials. , 2016, The New England journal of medicine.

[3]  David Moher,et al.  Increasing value and reducing waste in biomedical research: who's listening? , 2016, The Lancet.

[4]  Florian Gerber,et al.  gsbDesign: An R Package for Evaluating the Operating Characteristics of a Group Sequential Bayesian Design , 2016 .

[5]  Gernot Wassmer,et al.  Group Sequential and Confirmatory Adaptive Designs in Clinical Trials , 2016 .

[6]  A. Majid,et al.  The Potential of Adaptive Design in Animal Studies , 2015, International journal of molecular sciences.

[7]  I. Aban,et al.  Statistical considerations for preclinical studies , 2015, Experimental Neurology.

[8]  D. Cressey UK funders demand strong statistics for animal studies , 2015, Nature.

[9]  R. Lanfear,et al.  The Extent and Consequences of P-Hacking in Science , 2015, PLoS biology.

[10]  J. Ioannidis,et al.  Reproducibility in Science: Improving the Standard for Basic and Preclinical Research , 2015, Circulation research.

[11]  Jason A. Miranda,et al.  A Preclinical Physiological Assay to Test Modulation of Knee Joint Pain in the Spinal Cord: Effects of Oxycodone and Naproxen , 2014, PloS one.

[12]  Jeffrey N. Rouder,et al.  Why Hypothesis Tests Are Essential for Psychological Science , 2014, Psychological science.

[13]  D. Lakens Performing High-Powered Studies Efficiently with Sequential Analyses , 2014 .

[14]  Heinz Schmidli,et al.  A practical guide to Bayesian group sequential designs , 2014, Pharmaceutical statistics.

[15]  José Pinheiro,et al.  Practical Considerations for Adaptive Trial Design and Implementation , 2014 .

[16]  Brian A. Nosek,et al.  Power failure: why small sample size undermines the reliability of neuroscience , 2013, Nature Reviews Neuroscience.

[17]  Kristin L. Sainani The Problem of Multiple Testing , 2009, PM & R : the journal of injury, function, and rehabilitation.

[18]  I. Cuthill,et al.  Survey of the Quality of Experimental Design, Statistical Analysis and Reporting of Research Using Animals , 2009, PloS one.

[19]  K. Schulz,et al.  Multiplicity in randomised trials II: subgroup and interim analyses , 2005, The Lancet.

[20]  M. A. Best Bayesian Approaches to Clinical Trials and Health‐Care Evaluation , 2005 .

[21]  D L DeMets,et al.  Interim analysis: the alpha spending function approach. , 1994, Statistics in medicine.

[22]  S J Pocock,et al.  Interim analyses for randomized clinical trials: the group sequential approach. , 1982, Biometrics.

[23]  P. O'Brien,et al.  A multiple testing procedure for clinical trials. , 1979, Biometrics.

[24]  P. Armitage,et al.  Design and analysis of randomized clinical trials requiring prolonged observation of each patient. I. Introduction and design. , 1976, British Journal of Cancer.

[25]  J. Haybittle,et al.  Repeated assessment of results in clinical trials of cancer treatment. , 1971, The British journal of radiology.

[26]  D. Lindley The Use of Prior Probability Distributions in Statistical Inference and Decisions , 1961 .