Developing methods for the overarching synthesis of quantitative and qualitative evidence: The interweave synthesis approach

The incorporation of evidence derived from multiple research designs into one single synthesis can enhance the utility of systematic reviews making them more worthwhile, useful, and insightful. Methodological guidance for mixed‐methods synthesis continues to emerge and evolve but broadly involves a sequential, parallel, or convergent approach according to the degree of independence between individual syntheses before they are combined. We present two case studies in which we used novel and innovative methods to draw together the findings from individual but related quantitative and qualitative syntheses to aid interpretation of the overall evidence base. Our approach moved beyond making a choice between parallel, sequential, or convergent methods to interweave the findings of individual reviews and offers three key innovations to mixed‐methods synthesis methods: The use of intersubjective questions to understand the findings of the individual reviews through different lenses, Immersion of key reviewers in the entirety of the evidence base, and Commencing the process during the final stages of the synthesis of individual reviews, at a point where reviewers are developing an understanding of initial findings. Underlying our approach is the process of exploration and identification of links between and across review findings, an approach that is fundamental to all evidence syntheses but usually occurs at the level of the study. Adapting existing methods for exploring and identifying patterns and links between and across studies to interweave the findings between and across reviews may prove valuable.

[1]  Susan L Norris,et al.  Qualitative evidence to improve guidelines and health decision-making , 2018, Bulletin of the World Health Organization.

[2]  Sandy Oliver,et al.  Integrating qualitative research with trials in systematic reviews , 2004, BMJ : British Medical Journal.

[3]  Rebecca Rees,et al.  Weight management programmes: Re‐analysis of a systematic review to identify pathways to effectiveness , 2018, Health expectations : an international journal of public participation in health care and health policy.

[4]  Sharon E. Straus,et al.  Knowledge synthesis methods for integrating qualitative and quantitative data: a scoping review reveals poor operationalization of the methodological steps. , 2016, Journal of clinical epidemiology.

[5]  Kate Flemming,et al.  Cochrane Qualitative and Implementation Methods Group guidance series-paper 5: methods for integrating qualitative and implementation evidence within intervention effectiveness reviews. , 2017, Journal of clinical epidemiology.

[6]  Stuart Logan,et al.  Non-Pharmacological Interventions for ADHD in School Settings: An Overarching Synthesis of Systematic Reviews , 2019, Journal of attention disorders.

[7]  Ray Pawson,et al.  RAMESES publication standards: realist syntheses , 2013, BMC Medicine.

[8]  Terri D. Pigott,et al.  Identifying, documenting, and examining heterogeneity in systematic reviews of complex interventions. , 2013, Journal of clinical epidemiology.

[9]  Jane Noyes,et al.  Synthesizing evidence on complex interventions: how meta-analytical, qualitative, and mixed-method approaches can contribute. , 2013, Journal of clinical epidemiology.

[10]  N. Fielding,et al.  Going out on a Limb , 2009 .

[11]  Hilary Thomson,et al.  Lack of transparency in reporting narrative synthesis of quantitative data: a methodological assessment of systematic reviews , 2019, Journal of clinical epidemiology.

[12]  Susan Michie,et al.  A research and development agenda for systematic reviews that ask complex questions about complex interventions. , 2013, Journal of clinical epidemiology.

[13]  M. Sandelowski,et al.  Defining and Designing Mixed Research Synthesis Studies. , 2006, Research in the schools : a nationally refereed journal sponsored by the Mid-South Educational Research Association and the University of Alabama.

[14]  Ginny Brunton,et al.  Using qualitative comparative analysis (QCA) in systematic reviews of complex interventions: a worked example , 2014, Systematic Reviews.

[15]  James Thomas,et al.  Developing and Optimising the Use of Logic Models in Systematic Reviews: Exploring Practice and Good Practice in the Use of Programme Theory in Reviews , 2015, PloS one.

[16]  A. O’Cathain,et al.  Three techniques for integrating data in mixed methods studies , 2010, BMJ : British Medical Journal.

[17]  Susan Michie,et al.  Investigating complexity in systematic reviews of interventions by using a spectrum of methods. , 2013, Journal of clinical epidemiology.

[18]  Hilary Thomson,et al.  Improving Conduct and Reporting of Narrative Synthesis of Quantitative Data (ICONS-Quant): protocol for a mixed methods study to develop a reporting guideline , 2018, BMJ Open.

[19]  Mona Nasser,et al.  What are funders doing to minimise waste in research? , 2017, The Lancet.

[20]  Adam Fletcher,et al.  Teenage pregnancy and social disadvantage: systematic review integrating controlled trials and qualitative studies , 2009, BMJ : British Medical Journal.

[21]  Sandy Oliver,et al.  Using qualitative synthesis to explore heterogeneity of complex interventions , 2011, BMC medical research methodology.

[22]  Stuart Logan,et al.  Non-pharmacological interventions for attention-deficit/hyperactivity disorder (ADHD) delivered in school settings: systematic reviews of quantitative and qualitative research. , 2015, Health technology assessment.

[23]  A. Harden,et al.  Methods for the thematic synthesis of qualitative research in systematic reviews , 2008, BMC medical research methodology.

[24]  Pierre Pluye,et al.  Combining the power of stories and the power of numbers: mixed methods research and mixed studies reviews. , 2014, Annual review of public health.

[25]  Andrew Booth,et al.  Applying GRADE-CERQual to qualitative evidence synthesis findings: introduction to the series , 2018, Implementation Science.

[26]  Jennie Popay,et al.  Directly observed therapy and tuberculosis: how can a systematic review of qualitative research contribute to improving services? A qualitative meta-synthesis. , 2007, Journal of advanced nursing.

[27]  Mary Dixon-Woods,et al.  Synthesising qualitative and quantitative evidence: a review of possible methods. , 2005, Journal of health services research & policy.

[28]  R. Tibshirani,et al.  Increasing value and reducing waste in research design, conduct, and analysis , 2014, The Lancet.

[29]  Christopher Carroll,et al.  Qualitative evidence synthesis to improve implementation of clinical guidelines , 2017, British Medical Journal.

[30]  Kate Flemming,et al.  Cochrane Qualitative and Implementation Methods Group guidance series-paper 2: methods for question formulation, searching, and protocol development for qualitative evidence synthesis. , 2017, Journal of clinical epidemiology.

[31]  Jo Rycroft-Malone,et al.  Intervention Now to Eliminate Repeat Unintended Pregnancy in Teenagers (INTERUPT): a systematic review of intervention effectiveness and cost-effectiveness, and qualitative and realist synthesis of implementation factors and user engagement. , 2016, Health technology assessment.

[32]  John P. A. Ioannidis,et al.  Research: increasing value, reducing waste 2 , 2014 .

[33]  Michael Nunns,et al.  Interventions to improve the mental health of children and young people with long-term physical conditions: linked evidence syntheses. , 2019, Health technology assessment.

[34]  Stuart Logan,et al.  Improving the mental health of children and young people with long term conditions: Linked evidence syntheses , 2019 .

[35]  Alfonso Montuori,et al.  Complexity and Transdisciplinarity: Reflections on Theory and Practice , 2013 .

[36]  Meera Viswanathan,et al.  AHRQ series on complex intervention systematic reviews-paper 4: selecting analytic approaches. , 2017, Journal of clinical epidemiology.

[37]  Susan Salmond,et al.  A mixed-methods approach to systematic reviews. , 2015, International journal of evidence-based healthcare.

[38]  T. Greenhalgh,et al.  Realist review - a new method of systematic review designed for complex policy interventions , 2005, Journal of health services research & policy.

[39]  P. Pluye,et al.  Convergent and sequential synthesis designs: implications for conducting and reporting systematic reviews of qualitative and quantitative evidence , 2017, Systematic Reviews.