Taking account of context in systematic reviews and guidelines considering a complexity perspective

Systematic review teams and guideline development groups face considerable challenges when considering context within the evidence production process. Many complex interventions are context-dependent and are frequently evaluated within considerable contextual variation and change. This paper considers the extent to which current tools used within systematic reviews and guideline development are suitable in meeting these challenges. The paper briefly reviews strengths and weaknesses of existing approaches to specifying context. Illustrative tools are mapped to corresponding stages of the systematic review process. Collectively, systematic review and guideline production reveals a rich diversity of frameworks and tools for handling context. However, current approaches address only specific elements of context, are derived from primary studies which lack information or have not been tested within systematic reviews. A hypothetical example is used to illustrate how context could be integrated throughout the guideline development process. Guideline developers and evidence synthesis organisations should select an appropriate level of contextual detail for their specific guideline that is parsimonious and yet sensitive to health systems contexts and the values, preferences and needs of their target populations.

[1]  Jane Noyes,et al.  Implications of a complexity perspective for systematic reviews and guideline development in health decision making , 2019, BMJ Global Health.

[2]  Amal Mudallali.  Statement , 1988, Definitions.

[3]  Gemma Carey,et al.  Systems science and systems thinking for public health: a systematic review of the field , 2015, BMJ Open.

[4]  Gerald Gartlehner,et al.  [GRADE guidelines: 8. Rating the quality of evidence - indirectness]. , 2012, Zeitschrift fur Evidenz, Fortbildung und Qualitat im Gesundheitswesen.

[5]  Andrew D Oxman,et al.  SUPPORT Tools for evidence-informed health Policymaking (STP) 9: Assessing the applicability of the findings of a systematic review , 2009, Health research policy and systems.

[6]  Andrew Booth,et al.  Clear and present questions: formulating questions for evidence based practice , 2006, Libr. Hi Tech.

[7]  Spyridon N Papageorgiou,et al.  Registration in the international prospective register of systematic reviews (PROSPERO) of systematic review protocols was associated with increased review quality. , 2018, Journal of clinical epidemiology.

[8]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. , 2010, International journal of surgery.

[9]  H. Schünemann,et al.  [GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction.] , 2017, Recenti progressi in medicina.

[10]  P. Rothwell,et al.  External validity of randomised controlled trials: “To whom do the results of this trial apply?” , 2005, The Lancet.

[11]  Özge Tunçalp,et al.  Programme Reporting Standards (PRS) for improving the reporting of sexual, reproductive, maternal, newborn, child and adolescent health programmes , 2017, BMC Medical Research Methodology.

[12]  Anne MacFarlane,et al.  Assessing the facilitators and barriers of interdisciplinary team working in primary care using normalisation process theory: An integrative review , 2017, PloS one.

[13]  Elie A Akl,et al.  Current experience with applying the GRADE approach to public health interventions: an empirical study , 2013, BMC Public Health.

[14]  Jane Noyes,et al.  Synthesizing evidence on complex interventions: how meta-analytical, qualitative, and mixed-method approaches can contribute. , 2013, Journal of clinical epidemiology.

[15]  Kate Flemming,et al.  Qualitative evidence synthesis for complex interventions and guideline development: clarification of the purpose, designs and relevant methods , 2019, BMJ Global Health.

[16]  Andrew D Oxman,et al.  SUPPORT Tools for evidence-informed health Policymaking (STP) , 2009, Health research policy and systems.

[17]  Andrew Booth,et al.  Towards a methodology for cluster searching to provide conceptual and contextual “richness” for systematic reviews of complex interventions: case study (CLUSTER) , 2013, BMC Medical Research Methodology.

[18]  Meera Viswanathan,et al.  AHRQ series on complex intervention systematic reviews-paper 6: PRISMA-CI extension statement and checklist. , 2017, Journal of clinical epidemiology.

[19]  A. O’Cathain,et al.  Process evaluation of complex interventions: Medical Research Council guidance , 2015, BMJ : British Medical Journal.

[20]  Margarete Sandelowski,et al.  Text-in-context: a method for extracting findings in mixed-methods mixed research synthesis studies. , 2013, Journal of advanced nursing.

[21]  D. Tranfield,et al.  Developing Design Propositions through Research Synthesis , 2008 .

[22]  Birte Snilstveit,et al.  Evidence & Gap Maps: A tool for promoting evidence informed policy and strategic research agendas. , 2016, Journal of clinical epidemiology.

[23]  Elie A Akl,et al.  A methodological survey identified eight proposed frameworks for the adaptation of health related guidelines. , 2017, Journal of clinical epidemiology.

[24]  Kate Flemming,et al.  Synthesising quantitative and qualitative evidence to inform guidelines on complex interventions: clarifying the purposes, designs and outlining some methods , 2019, BMJ Global Health.

[25]  Jay Shaw,et al.  Conditions : A Metanarrative Review , 2017 .

[26]  Susan L Norris,et al.  The WHO-INTEGRATE evidence to decision framework version 1.0: integrating WHO norms and values and a complexity perspective , 2019, BMJ Global Health.

[27]  Jo Rycroft-Malone,et al.  Realist synthesis: illustrating the method for implementation research , 2012, Implementation Science.

[28]  D. Moher,et al.  CONSORT 2010 statement: Updated guidelines for reporting parallel group randomised trials , 2010, Journal of pharmacology & pharmacotherapeutics.

[29]  G. Moore,et al.  What theory, for whom and in which context? Reflections on the application of theory in the development and evaluation of complex population health interventions , 2016, SSM - population health.

[30]  Dawid Pieper,et al.  HEALTH TECHNOLOGY ASSESSMENT OF PUBLIC HEALTH INTERVENTIONS: A SYNTHESIS OF METHODOLOGICAL GUIDANCE , 2017, International Journal of Technology Assessment in Health Care.

[31]  Claire Glenton,et al.  Qualitative "trial-sibling" studies and "unrelated" qualitative studies contributed to complex intervention reviews. , 2016, Journal of clinical epidemiology.

[32]  Susan Michie,et al.  Assessing the complexity of interventions within systematic reviews: development, content and use of a new tool (iCAT_SR) , 2017, BMC Medical Research Methodology.

[33]  J. Eisenberg,et al.  Globalize the evidence, localize the decision: evidence-based medicine and international diversity. , 2002, Health affairs.

[34]  Andrew Booth,et al.  Applying GRADE-CERQual to qualitative evidence synthesis findings–paper 6: how to assess relevance of the data , 2018, Implementation Science.

[35]  P. Shekelle,et al.  Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation , 2016, British Medical Journal.

[36]  G. Guyatt,et al.  GRADE: an emerging consensus on rating quality of evidence and strength of recommendations , 2008, BMJ : British Medical Journal.

[37]  Sam Porter,et al.  The use and limitation of realistic evaluation as a tool for evidence-based practice: a critical realist perspective. , 2012, Nursing inquiry.

[38]  A. Weil,et al.  Diffusion of Innovation , 2020, The International Encyclopedia of Media Psychology.

[39]  Natalie Taylor,et al.  Identifying the barriers and enablers for a triage, treatment, and transfer clinical intervention to manage acute stroke patients in the emergency department: a systematic review using the theoretical domains framework (TDF) , 2016, Implementation Science.

[40]  Anne Mills,et al.  What do we mean by rigorous health-systems research? , 2008, The Lancet.

[41]  Naoki Ikegami,et al.  The World Health Report 2008 - Primary Healthcare: How Wide Is the Gap between Its Agenda and Implementation in 12 High-Income Health Systems? , 2012, Healthcare policy = Politiques de sante.

[42]  Mary Butler,et al.  AHRQ series on complex intervention systematic reviews-paper 7: PRISMA-CI elaboration and explanation. , 2017, Journal of clinical epidemiology.

[43]  Andrew Booth,et al.  Current use was established and Cochrane guidance on selection of social theories for systematic reviews of complex interventions was developed. , 2016, Journal of clinical epidemiology.

[44]  Helen E. D. Burchett,et al.  How do we know when research from one setting can be useful in another? A review of external validity, applicability and transferability frameworks , 2011, Journal of health services research & policy.

[45]  Gerald Gartlehner,et al.  [GRADE guidelines 15: going from evidence to recommendation - determinants of a recommendation's direction and strength]. , 2014, Zeitschrift fur Evidenz, Fortbildung und Qualitat im Gesundheitswesen.

[46]  J. Wyatt,et al.  Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide , 2014, BMJ : British Medical Journal.

[47]  Jemma Hawkins,et al.  All interventions are complex, but some are more complex than others: using iCAT_SR to assess complexity. , 2017, The Cochrane database of systematic reviews.

[48]  Mark Petticrew,et al.  When are complex interventions 'complex'? When are simple interventions 'simple'? , 2011, European journal of public health.

[49]  Andrew Booth,et al.  Making sense of complexity in context and implementation: the Context and Implementation of Complex Interventions (CICI) framework , 2017, Implementation Science.

[50]  Tim Evans,et al.  Applying an equity lens to interventions: using PROGRESS ensures consideration of socially stratifying factors to illuminate inequities in health. , 2014, Journal of clinical epidemiology.

[51]  K. Hopayian,et al.  Outside the box: why are Cochrane reviews so boring? , 2012, The British journal of general practice : the journal of the Royal College of General Practitioners.

[52]  Devon L. Greyson,et al.  A critical examination of representations of context within research on population health interventions , 2016 .

[53]  Elizabeth Waters,et al.  Assessing the applicability of findings in systematic reviews of complex interventions can enhance the utility of reviews for decision making. , 2013, Journal of clinical epidemiology.

[54]  Niranjan Kissoon,et al.  Theoretical domains framework to assess barriers to change for planning health care quality interventions: a systematic literature review , 2016, Journal of multidisciplinary healthcare.

[55]  S. Murray,et al.  National health policy-makers’ views on the clarity and utility of Countdown to 2015 country profiles and reports: findings from two exploratory qualitative studies , 2014, Health Research Policy and Systems.

[56]  Deborah M Caldwell,et al.  Synthesising quantitative evidence in systematic reviews of complex health interventions , 2019, BMJ Global Health.

[57]  Nancy D Berkman,et al.  AHRQ series on complex intervention systematic reviews-paper 3: adapting frameworks to develop protocols. , 2017, Journal of clinical epidemiology.

[58]  Alonso Carrasco-Labra,et al.  GRADE Evidence to Decision (EtD) frameworks for adoption, adaptation, and de novo development of trustworthy recommendations: GRADE-ADOLOPMENT. , 2017, Journal of clinical epidemiology.

[59]  Paul Montgomery,et al.  The Oxford Implementation Index: a new tool for incorporating implementation data into systematic reviews and meta-analyses. , 2013, Journal of clinical epidemiology.

[60]  Kate Flemming,et al.  Formulating questions to explore complex interventions within qualitative evidence synthesis , 2019, BMJ Global Health.

[61]  Ross S Bailie,et al.  Making systematic reviews more useful for policy-makers. , 2005, Bulletin of the World Health Organization.

[62]  M. Petticrew,et al.  Systematic Reviews in the Social Sciences: A Practical Guide , 2005 .

[63]  R. Glasgow,et al.  The RE-AIM framework: a systematic review of use over time. , 2013, American journal of public health.

[64]  Alan Shiell,et al.  Theorising Interventions as Events in Systems , 2009, American journal of community psychology.

[65]  C. Dowrick,et al.  Complex interventions , 2022, International Review of Sport and Exercise Psychology.

[66]  Blake Poland,et al.  Settings for Health Promotion: An Analytic Framework to Guide Intervention Design and Implementation , 2009, Health promotion practice.

[67]  Brenton G. Abadie,et al.  A systematic review of the use of the Consolidated Framework for Implementation Research , 2015, Implementation Science.

[68]  Earl L. Taylor,et al.  The Diffusion of Innovation , 2013 .

[69]  Paul Wilson,et al.  Getting messier with TIDieR: embracing context and complexity in intervention reporting , 2018, BMC Medical Research Methodology.

[70]  Andrew Booth,et al.  Context and implementation: A concept analysis towards conceptual maturity. , 2015, Zeitschrift fur Evidenz, Fortbildung und Qualitat im Gesundheitswesen.

[71]  Mohammed T Ansari,et al.  The GRADE Working Group clarifies the construct of certainty of evidence. , 2017, Journal of clinical epidemiology.

[72]  Bruce Guthrie,et al.  Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting , 2013, Trials.

[73]  John N Lavis,et al.  Developing a workbook to support the contextualisation of global health systems guidance: a case study identifying steps and critical factors for success in this process at WHO , 2018, Health Research Policy and Systems.

[74]  Andrew Booth,et al.  Using Qualitative Evidence in Decision Making for Health and Social Interventions: An Approach to Assess Confidence in Findings from Qualitative Evidence Syntheses (GRADE-CERQual) , 2015, PLoS medicine.

[75]  G H Guyatt,et al.  A Consumer's Guide to Subgroup Analyses , 1992, Annals of Internal Medicine.

[76]  Jayne Webster,et al.  Methods for evaluating delivery systems for scaling-up malaria control intervention , 2010, BMC health services research.

[77]  Ray Pawson,et al.  Guidance on guidelines: Understanding the evidence on the uptake of health care guidelines , 2018, Journal of evaluation in clinical practice.

[78]  J. Lowery,et al.  Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science , 2009, Implementation science : IS.

[79]  S. Oliver,et al.  Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ , 2012, BMC Medical Research Methodology.

[80]  Gilda Piaggio,et al.  Alternative versus standard packages of antenatal care for low-risk pregnancy. , 2015, The Cochrane database of systematic reviews.

[81]  Peter Tugwell,et al.  Adolopment - a new term added to the Clinical Epidemiology Lexicon. , 2017, Journal of clinical epidemiology.

[82]  G. Antes,et al.  CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials , 2011 .

[83]  Iveta Simera,et al.  A history of the evolution of guidelines for reporting medical research: the long road to the EQUATOR Network , 2016, Journal of the Royal Society of Medicine.

[84]  Andrew D Oxman,et al.  SUPPORT Tools for evidence-informed Policymaking in health 11: Finding and using evidence about local conditions , 2009, Health research policy and systems.

[85]  E. V. van Sluijs,et al.  Exploring equity in primary-care-based physical activity interventions using PROGRESS-Plus: a systematic review and evidence synthesis , 2016, International Journal of Behavioral Nutrition and Physical Activity.

[86]  Julie Glanville,et al.  How to formulate research recommendations , 2006, BMJ : British Medical Journal.

[87]  Steven Cummins,et al.  Taking account of context in population health intervention research: guidance for producers, users and funders of research , 2018 .

[88]  Andrew Booth,et al.  Towards a taxonomy of logic models in systematic reviews and health technology assessments: A priori, staged, and iterative approaches , 2018, Research synthesis methods.

[89]  Christian D Helfrich,et al.  A critical synthesis of literature on the promoting action on research implementation in health services (PARIHS) framework , 2010, Implementation Science : IS.

[90]  Graham Moore,et al.  Contextual issues and qualitative research , 2016 .

[91]  Kate Flemming,et al.  Guidance on choosing qualitative evidence synthesis methods for use in health technology assessments of complex interventions , 2016 .

[92]  Robert L Kane,et al.  AHRQ series on complex intervention systematic reviews-paper 2: defining complexity, formulating scope, and questions. , 2017, Journal of clinical epidemiology.