Bayesian design and analysis of external pilot trials for complex interventions

External pilot trials of complex interventions are used to help determine if and how a confirmatory trial should be undertaken, providing estimates of parameters such as recruitment, retention, and adherence rates. The decision to progress to the confirmatory trial is typically made by comparing these estimates to pre-specified thresholds known as progression criteria, although the statistical properties of such decision rules are rarely assessed. Such assessment is complicated by several methodological challenges, including the simultaneous evaluation of multiple endpoints, complex multi-level models, small sample sizes, and uncertainty in nuisance parameters. In response to these challenges, we describe a Bayesian approach to the design and analysis of external pilot trials. We show how progression decisions can be made by minimizing the expected value of a loss function, defined over the whole parameter space to allow for preferences and trade-offs between multiple parameters to be articulated and used in the decision-making process. The assessment of preferences is kept feasible by using a piecewise constant parametrization of the loss function, the parameters of which are chosen at the design stage to lead to desirable operating characteristics. We describe a flexible, yet computationally intensive, nested Monte Carlo algorithm for estimating operating characteristics. The method is used to revisit the design of an external pilot trial of a complex intervention designed to increase the physical activity of care home residents.

[1]  Sin-Ho Jung,et al.  Admissible two‐stage designs for phase II cancer clinical trials , 2004, Statistics in medicine.

[2]  Jeremy E. Oakley,et al.  Estimating the Expected Value of Sample Information Using the Probabilistic Sensitivity Analysis Sample , 2015, Medical decision making : an international journal of the Society for Medical Decision Making.

[3]  R. H. Browne.  On the use of a pilot sample for sample size determination. , 1995, Statistics in medicine.

[4]  Meinhard Kieser,et al.  Utility‐based optimization of phase II/III programs , 2016, Statistics in medicine.

[5]  Nigel Stallard,et al.  Optimal sample sizes for phase II clinical trials and pilot studies , 2012, Statistics in medicine.

[6]  R. Crutzen,et al.  A simple formula for the calculation of sample size in pilot studies. , 2015, Journal of clinical epidemiology.

[7]  Julius Sim,et al.  The size of a pilot study for a clinical trial should be calculated in relation to considerations of precision and efficiency. , 2012, Journal of clinical epidemiology.

[8]  Meinhard Kieser,et al.  Sample size planning for phase II trials based on success probabilities for phase III , 2015, Pharmaceutical statistics.

[9]  M. Campbell,et al.  Defining Feasibility and Pilot Studies in Preparation for Randomised Controlled Trials: Development of a Conceptual Framework , 2016, PloS one.

[10]  Nicky Best,et al.  Practical experiences of adopting assurance as a quantitative framework to support decision making in drug development , 2018, Pharmaceutical statistics.

[11]  C. Gamble,et al.  Informing efficient randomised controlled trials: exploration of challenges in developing progression criteria for internal pilot studies , 2017, BMJ Open.

[12]  Gordon Graham,et al.  Bayesian sample size for exploratory clinical trials incorporating historical data , 2008, Statistics in medicine.

[13]  M. Petticrew,et al.  Developing and evaluating complex interventions: the new Medical Research Council guidance , 2008, BMJ : British Medical Journal.

[14]  M. Campbell,et al.  CONSORT 2010 statement: extension to randomised pilot and feasibility trials , 2016, British Medical Journal.

[15]  G. Lancaster,et al.  How big should the pilot study for my cluster randomised trial be? , 2016, Statistical methods in medical research.

[16]  M. Segal,et al.  Simple, Defensible Sample Sizes Based on Cost Efficiency , 2008, Biometrics.

[17]  Adrian P Mander,et al.  Admissible two‐stage designs for phase II cancer clinical trials that incorporate the expected sample size under the alternative hypothesis , 2012, Pharmaceutical statistics.

[18]  M. Teare,et al.  Sample size requirements to estimate key design parameters from external pilot randomised controlled trials: a simulation study , 2014, Trials.

[19]  Phil Woodward,et al.  Advantages of a wholly Bayesian approach to assessing efficacy in early drug development: a case study , 2015, Pharmaceutical statistics.

[20]  Nicola J Cooper,et al.  Evidence‐based sample size calculations based upon updated meta‐analysis , 2007, Statistics in medicine.

[21]  Yves Deville,et al.  DiceKriging, DiceOptim: Two R Packages for the Analysis of Computer Experiments by Kriging-Based Metamodeling and Optimization , 2012 .

[22]  P. Williamson,et al.  Design and analysis of pilot studies: recommendations for good practice. , 2004, Journal of evaluation in clinical practice.

[23]  Jeremy E. Oakley,et al.  Estimating Multiparameter Partial Expected Value of Perfect Information from a Probabilistic Sensitivity Analysis Sample , 2013, Medical decision making : an international journal of the Society for Medical Decision Making.

[24]  Nicky Best,et al.  Better decision making in drug development through adoption of formal prior elicitation , 2017, Pharmaceutical statistics.

[25]  Fei Wang,et al.  A simulation-based approach to Bayesian sample size determination for performance under a given model and for separating models , 2002 .

[26]  D J Spiegelhalter,et al.  Bayesian methods for cluster randomized trials with continuous responses. , 2001, Statistics in medicine.

[27]  Anthony O'Hagan,et al.  Assurance in clinical trial design , 2005 .

[28]  M. G. Pittau,et al.  A weakly informative default prior distribution for logistic and other regression models , 2008, 0901.4011.

[29]  D. Lindley.  The choice of sample size , 1997 .

[30]  Peter Bacchetti,et al.  Current sample size conventions: Flaws, harms, and alternatives , 2010, BMC medicine.

[31]  Steven A Julious,et al.  Estimating the sample size for a pilot randomised trial to minimise the overall trial sample size for the external pilot and main trial for a continuous outcome variable , 2015, Statistical methods in medical research.

[32]  Donald R. Jones,et al.  A Taxonomy of Global Optimization Methods Based on Response Surfaces , 2001, J. Glob. Optim..

[33]  Steven A. Julious,et al.  Sample size of 12 per group rule of thumb for a pilot study , 2005 .

[34]  David B. Wolfson,et al.  Interval‐based versus decision theoretic criteria for the choice of sample size , 1997 .

[35]  Jeremy E. Oakley,et al.  Uncertain Judgements: Eliciting Experts' Probabilities , 2006 .

[36]  F. B. Vernadat,et al.  Decisions with Multiple Objectives: Preferences and Value Tradeoffs , 1994 .

[37]  R. L. Keeney,et al.  Decisions with Multiple Objectives: Preferences and Value Trade-Offs , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[38]  A. Forster,et al.  Research Exploring Physical Activity in Care Homes (REACH): study protocol for a randomised controlled trial , 2017, Trials.

[39]  Duncan T. Wilson,et al.  Statistical challenges in assessing potential efficacy of complex interventions in pilot or feasibility studies , 2016, Statistical methods in medical research.

[40]  Lisa V Hampson,et al.  A framework for prospectively defining progression rules for internal pilot studies monitoring recruitment , 2018, Statistical methods in medical research.

[41]  S. Julious,et al.  Are pilot trials useful for predicting randomisation and attrition rates in definitive studies: A review of publicly funded trials , 2018, Clinical trials.