The role of pragmatism in explaining heterogeneity in meta-analyses of randomised trials: a protocol for a cross-sectional methodological review

Introduction There has been increasing interest in pragmatic trials methodology. As a result, tools such as the Pragmatic-Explanatory Continuum Indicator Summary-2 (PRECIS-2) are being used prospectively to help researchers design randomised controlled trials (RCTs) within the pragmatic-explanatory continuum. There may be value in applying the PRECIS-2 tool retrospectively in a systematic review setting as it could provide important information about how to pool data based on the degree of pragmatism. Objectives To investigate the role of pragmatism as a source of heterogeneity in systematic reviews by (1) identifying systematic reviews with meta-analyses of RCTs that have moderate to high heterogeneity, (2) applying PRECIS-2 to RCTs of systematic reviews, (3) evaluating the inter-rater reliability of PRECIS-2, (4) determining how much of this heterogeneity may be explained by pragmatism. Methods A cross-sectional methodological review will be conducted on systematic reviews of RCTs published in the Cochrane Library from 1 January 2014 to 1 January 2017. Included systematic reviews will have a minimum of 10 RCTs in the meta-analysis of the primary outcome and moderate to substantial heterogeneity (I2≥50%). Of the eligible systematic reviews, a random selection of 10 will be included for quantitative evaluation. In each systematic review, RCTs will be scored using the PRECIS-2 tool, in duplicate. Agreement between raters will be measured using the intraclass correlation coefficient. Subgroup analyses and meta-regression will be used to evaluate how much variability in the primary outcome may be due to pragmatism. Dissemination This review will be among the first to evaluate the PRECIS-2 tool in a systematic review setting. Results from this research will provide inter-rater reliability information about PRECIS-2 and may be used to provide methodological guidance when dealing with pragmatism in systematic reviews and subgroup considerations. On completion, this review will be submitted to a peer-reviewed journal for publication.

[1]  S. Zhan,et al.  [The PRECIS-2 tool: designing trials that are fit for purpose]. , 2018, Zhonghua liu xing bing xue za zhi = Zhonghua liuxingbingxue zazhi.

[2]  P. Donnan,et al.  The PRECIS-2 tool has good interrater reliability and modest discriminant validity. , 2017, Journal of clinical epidemiology.

[3]  M. Zwarenstein,et al.  PRECIS-2 helps researchers design more applicable RCTs while CONSORT Extension for Pragmatic Trials helps knowledge users decide whether to apply them. , 2017, Journal of clinical epidemiology.

[4]  P. Donnan,et al.  The PRECIS-2 tool: designing trials that are fit for purpose , 2015, BMJ : British Medical Journal.

[5]  E. Waters,et al.  Exploring the pragmatic and explanatory study design on outcomes of systematic reviews of public health interventions: a case study on obesity prevention trials. , 2014, Journal of public health.

[6]  Richard Smith,et al.  The Cochrane Collaboration at 20 , 2013, BMJ.

[7]  R. Lüdtke,et al.  How Well Do Randomized Trials Inform Decision Making: Systematic Review Using Comparative Effectiveness Research Measures on Acupuncture for Back Pain , 2012, PloS one.

[8]  J André Knottnerus,et al.  Pragmatic vs. explanatory: an adaptation of the PRECIS tool helps to judge the applicability of systematic reviews for daily practice. , 2011, Journal of clinical epidemiology.

[9]  C. Adams,et al.  Pragmatic vs explanatory trials: the Pragmascope tool to help measure differences in protocols of mental health randomized controlled trials , 2011, Dialogues in clinical neuroscience.

[10]  N. Patsopoulos.  A pragmatic view on pragmatic trials , 2011, Dialogues in clinical neuroscience.

[11]  Ian Harvey,et al.  A pragmatic–explanatory continuum indicator summary (PRECIS): a tool to help trial designers , 2009, Canadian Medical Association Journal.

[12]  Ian Harvey,et al.  A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. , 2009, Journal of clinical epidemiology.

[13]  K. Lohr,et al.  A simple and valid tool distinguished efficacy from effectiveness studies. , 2006, Journal of clinical epidemiology.

[14]  Fujian Song,et al.  Quality of Cochrane reviews , 2002, BMJ : British Medical Journal.

[15]  Gordon H. Guyatt,et al.  Clinical Epidemiology: How to Do Clinical Practice Research , 1991 .

[16]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[17]  J. Lellouch,et al.  Explanatory and pragmatic attitudes in therapeutical trials. , 1967, Journal of chronic diseases.

[18]  Gerald Gartlehner,et al.  [GRADE guidelines: 7. Rating the quality of evidence - inconsistency]. , 2012, Zeitschrift fur Evidenz, Fortbildung und Qualitat im Gesundheitswesen.

[19]  S. Helgerson.  Clinical Epidemiology: How to Do Clinical Practice Research , 2006 .

[20]  E. Waters,et al.  Interventions for preventing obesity in children. , 2001, The Cochrane database of systematic reviews.