Guidance for conducting feasibility and pilot studies for implementation trials

Background Implementation trials aim to test the effects of implementation strategies on the adoption, integration or uptake of an evidence-based intervention within organisations or settings. Feasibility and pilot studies can assist with building and testing effective implementation strategies by helping to address uncertainties around design and methods, assessing potential implementation strategy effects and identifying potential causal mechanisms. This paper aims to provide broad guidance for the conduct of feasibility and pilot studies for implementation trials. Methods We convened a group with a mutual interest in the use of feasibility and pilot trials in implementation science including implementation and behavioural science experts and public health researchers. We conducted a literature review to identify existing recommendations for feasibility and pilot studies, as well as publications describing formative processes for implementation trials. In the absence of previous explicit guidance for the conduct of feasibility or pilot implementation trials specifically, we used the effectiveness-implementation hybrid trial design typology proposed by Curran and colleagues as a framework for conceptualising the application of feasibility and pilot testing of implementation interventions. We discuss and offer guidance regarding the aims, methods, design, measures, progression criteria and reporting for implementation feasibility and pilot studies. Conclusions This paper provides a resource for those undertaking preliminary work to enrich and inform larger scale implementation trials.

[1]  M. Campbell,et al.  Defining Feasibility and Pilot Studies in Preparation for Randomised Controlled Trials: Development of a Conceptual Framework , 2016, PloS one.

[2]  Elizabeth Murray,et al.  Standards for Reporting Implementation Studies (StaRI) Statement , 2017, British Medical Journal.

[3]  Jennifer Leeman,et al.  Beyond “implementation strategies”: classifying the full range of strategies used in implementation science and practice , 2017, Implementation Science.

[4]  M. Petticrew,et al.  Developing and evaluating complex interventions: the new Medical Research Council guidance , 2008, BMJ : British Medical Journal.

[5]  Gerjo Kok,et al.  Implementation Mapping: Using Intervention Mapping to Develop Implementation Strategies , 2019, Front. Public Health.

[6]  M. Campbell,et al.  What is a pilot or feasibility study? A review of current practice and editorial policy , 2010, BMC medical research methodology.

[7]  J. Donovan,et al.  Maximising the impact of qualitative research in feasibility studies for randomised controlled trials: guidance for researchers , 2015, Trials.

[8]  Graham Moore,et al.  Realist complex intervention science: Applying realist principles across all phases of the Medical Research Council framework for developing and evaluating complex interventions , 2016, Evaluation.

[9]  J. Sterne,et al.  The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials , 2011, BMJ : British Medical Journal.

[10]  Hilde van der Togt,et al.  Publisher's Note , 2003, J. Netw. Comput. Appl..

[11]  C. Moore,et al.  Recommendations for Planning Pilot Studies in Clinical and Translational Research , 2011, Clinical and translational science.

[12]  A. Ismaila,et al.  A tutorial on pilot studies: the what, why and how , 2010, BMC Medical Research Methodology.

[13]  Luke Wolfenden,et al.  Psychometric properties of implementation measures for public health and community settings and mapping of constructs against the Consolidated Framework for Implementation Research: a systematic review , 2016, Implementation Science.

[14]  M. Teare,et al.  Sample size requirements to estimate key design parameters from external pilot randomised controlled trials: a simulation study , 2014, Trials.

[15]  A. Kilbourne,et al.  An introduction to implementation science for the non-specialist , 2015, BMC psychology.

[16]  G. Moore,et al.  Exploratory studies to decide whether and how to proceed with full-scale evaluations of public health interventions: a systematic review of guidance , 2018, Pilot and Feasibility Studies.

[17]  G. Lancaster,et al.  Improving the efficiency of trials using innovative pilot designs: the next phase in the conduct and reporting of pilot and feasibility studies , 2017, Pilot and Feasibility Studies.

[18]  Andrew Booth,et al.  Implementation Science BioMed Central Debate A conceptual framework for implementation fidelity , 2007 .

[19]  M. Orrell,et al.  Acceptance checklist for clinical effectiveness pilot trials: a systematic approach , 2013, BMC Medical Research Methodology.

[20]  J. Durlak,et al.  Implementation Matters: A Review of Research on the Influence of Implementation on Program Outcomes and the Factors Affecting Implementation , 2008, American journal of community psychology.

[21]  Geoffrey M. Curran,et al.  Effectiveness-implementation Hybrid Designs: Combining Elements of Clinical Effectiveness and Implementation Research to Enhance Public Health Impact , 2012, Medical care.

[22]  S. Pocock,et al.  Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): Explanation and Elaboration , 2007, PLoS medicine.

[23]  Enola K. Proctor,et al.  A Compilation of Strategies for Implementing Clinical Innovations in Health and Mental Health , 2012, Medical care research and review : MCRR.

[24]  P. Williamson,et al.  Design and analysis of pilot studies: recommendations for good practice. , 2004, Journal of evaluation in clinical practice.

[25]  Kayne D. Mettert,et al.  An updated protocol for a systematic review of implementation-related measures , 2018, Systematic Reviews.

[26]  Luke Wolfenden,et al.  Implementation and scale-up of physical activity and behavioural nutrition interventions: an evaluation roadmap , 2019, International Journal of Behavioral Nutrition and Physical Activity.

[27]  B. Weiner,et al.  Advancing implementation science through measure development and evaluation: a study protocol , 2015, Implementation Science.

[28]  M. Morris,et al.  Beyond RCTs: Innovations in research design and methods to advance implementation science , 2017 .

[29]  Elizabeth A. Stuart,et al.  The Nonuse, Misuse, and Proper Use of Pilot Studies in Experimental Evaluation Research , 2017 .

[30]  A. O’Cathain,et al.  Process evaluation of complex interventions: Medical Research Council guidance , 2015, BMJ : British Medical Journal.

[31]  G. Aarons,et al.  Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors , 2010, Administration and Policy in Mental Health and Mental Health Services Research.

[32]  H. Kraemer,et al.  Caution regarding the use of pilot studies to guide power calculations for study proposals. , 2006, Archives of general psychiatry.

[33]  Candice Bowman,et al.  The role of formative evaluation in implementation research and the QUERI experience , 2006, Journal of General Internal Medicine.

[34]  Lisa V Hampson,et al.  A framework for prospectively defining progression rules for internal pilot studies monitoring recruitment , 2018, Statistical methods in medical research.

[35]  Cathie M Scott,et al.  Health Promotion Planning: An Educational and Ecological Approach (3rd ed) , 2001 .

[36]  Maria E Fernandez,et al.  How we design feasibility studies. , 2009, American journal of preventive medicine.

[37]  R. Pickering,et al.  Questions asked and answered in pilot and feasibility randomized controlled trials , 2011, BMC medical research methodology.

[38]  Sara J. Landes,et al.  Reprint of: An introduction to effectiveness-implementation hybrid designs , 2019, Psychiatry Research.

[39]  M. Harrison,et al.  Framework for planning and conducting pilot studies. , 2009, Ostomy/wound management.

[40]  Caitlin N. Dorsey,et al.  Psychometric assessment of three newly developed implementation outcome measures , 2017, Implementation Science.

[41]  S. Julious,et al.  An audit of sample sizes for pilot and feasibility trials being undertaken in the United Kingdom registered in the United Kingdom Clinical Research Network database , 2013, BMC Medical Research Methodology.

[42]  Ruth P Saunders,et al.  Developing a Process-Evaluation Plan for Assessing Health Promotion Program Implementation: A How-To Guide , 2005, Health promotion practice.

[43]  S. Hagen,et al.  A process for Decision-making after Pilot and feasibility Trials (ADePT): development following a feasibility study of a complex intervention for pelvic organ prolapse , 2013, Trials.

[44]  E. Proctor,et al.  Writing implementation research grant proposals: ten key ingredients , 2012, Implementation Science.

[45]  Trish Groves,et al.  Enhancing the quality and transparency of health research , 2008, BMJ : British Medical Journal.

[46]  H. Kraemer,et al.  The role and interpretation of pilot studies in clinical research. , 2011, Journal of psychiatric research.

[47]  Gabriele Meyer,et al.  Criteria for Reporting the Development and Evaluation of Complex Interventions in healthcare: revised guideline (CReDECI 2) , 2015, Trials.

[48]  J. Lowery,et al.  Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science , 2009, Implementation science : IS.

[49]  A FRAMEWORK FOR DEVELOPMENT AND EVALUATION OF RCTs FOR COMPLEX INTERVENTIONS TO IMPROVE HEALTH , 2000 .

[50]  S. Julious,et al.  Guidance for using pilot studies to inform the design of intervention trials with continuous outcomes , 2018, Clinical epidemiology.

[51]  G. Lancaster,et al.  How big should the pilot study for my cluster randomised trial be? , 2016, Statistical methods in medical research.

[52]  J. Grimshaw,et al.  Changing the behavior of healthcare professionals: the use of theory in promoting the uptake of research findings. , 2005, Journal of clinical epidemiology.

[53]  L. Palinkas,et al.  Peer-led healthy lifestyle program in supportive housing: study protocol for a randomized controlled trial , 2015, Trials.

[54]  Michael R. Kauth,et al.  Increasing the impact of randomized controlled trials: an example of a hybrid effectiveness-implementation design in psychotherapy research. , 2018, Translational behavioral medicine.

[55]  C. Gamble,et al.  Informing efficient randomised controlled trials: exploration of challenges in developing progression criteria for internal pilot studies , 2017, BMJ Open.

[56]  C. Donaldson,et al.  Optimisation of complex health interventions prior to a randomised controlled trial: a scoping review of strategies used , 2016, Pilot and Feasibility Studies.

[57]  J. Wyatt,et al.  Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide , 2014, BMJ : British Medical Journal.

[58]  E. Campbell,et al.  A cluster randomised trial of a telephone-based intervention for parents to increase fruit and vegetable consumption in their 3- to 5-year-old children: study protocol , 2010, BMC public health.

[59]  Alicia C. Bunger,et al.  Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda , 2010, Administration and Policy in Mental Health and Mental Health Services Research.

[60]  M. Campbell,et al.  CONSORT 2010 statement: extension to randomised pilot and feasibility trials , 2016, British Medical Journal.

[61]  S. Flottorp,et al.  Implementation science: a reappraisal of our journal mission and scope , 2015, Implementation Science.

[62]  Shannon Wiltsey Stirman,et al.  Development of a framework and coding system for modifications and adaptations of evidence-based interventions , 2013, Implementation Science.

[63]  J. Kirchner,et al.  A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project , 2015, Implementation Science.

[64]  Cara C. Lewis,et al.  From Classification to Causality: Advancing Understanding of Mechanisms of Change in Implementation Science , 2018, Front. Public Health.

[65]  Cynthia Vinson,et al.  Implementation science in cancer prevention and control: a decade of grant funding by the National Cancer Institute and future directions , 2015, Implementation Science.

[66]  Thomas M Vogt,et al.  Evaluating the impact of health promotion programs: using the RE-AIM framework to form summary measures for decision making involving complex issues. , 2006, Health education research.

[67]  Enola K Proctor,et al.  Implementation strategies: recommendations for specifying and reporting , 2013, Implementation Science.

[68]  D. Ward,et al.  A pilot randomised controlled trial of a web-based implementation intervention to increase child intake of fruit and vegetables within childcare centres , 2020, Pilot and Feasibility Studies.

[69]  Explore Configuring,et al.  A Simulation Study to , 2004 .

[70]  Caitlin N. Dorsey,et al.  Proceedings of the Fourth Biennial Conference of the Society for Implementation Research Collaboration (SIRC) 2017: implementation mechanisms: what makes implementation work and why? part 1 , 2018, Implementation Science.

[71]  D. Pérez,et al.  A modified theoretical framework to assess implementation fidelity of adaptive public health interventions , 2015, Implementation Science.

[72]  G. Meyer,et al.  Quality of reporting of complex healthcare interventions and applicability of the CReDECI list - a survey of publications indexed in PubMed , 2013, BMC Medical Research Methodology.

[73]  Jo Rycroft-Malone,et al.  An implementation research agenda , 2009, Implementation science : IS.