Methodological Issues in Combining Diverse Study Types in Systematic Reviews

In this paper we discuss mixing methods at the level of reviews of research, combining the findings of multiple, already existing, studies that are labelled broadly as using either ‘qualitative’ or ‘quantitative’ methods. We define systematic reviews and outline the ‘mixed methods’ we have developed for combining diverse study types within them. Traditional systematic reviews usually answer a single question, use one type of study and, hence, only require one method of synthesis to combine the findings of studies. Our methods involve conducting three types of synthesis: (1) a statistical meta‐analysis to pool trials of interventions tackling a particular health, social or educational problem; (2) a synthesis of studies examining people’s perspectives or experiences of that problem using qualitative analysis; and (3) a mixed methods synthesis bringing the products of (1) and (2) together. We discuss the strengths of these mixed methods at the review level, reflect on their lessons for the concept of ‘triangulation’ and raise questions about the utility of the terms ‘qualitative’ and ‘quantitative’ and ‘mixed methods’ for classifying different types of research.

[1]  John W. Creswell,et al.  Research Design: Qualitative, Quantitative, and Mixed Methods Approaches , 2010 .

[2]  T. Cook,et al.  Qualitative and quantitative methods in evaluation research , 1981 .

[3]  A. Bryman.  Quantity and quality in social research , 1988 .

[4]  G. Noblit,et al.  Meta-Ethnography: Synthesizing Qualitative Studies , 1988 .

[5]  M. Hammersley.  What's Wrong With Ethnography? , 1991 .

[6]  N. Denzin,et al.  Handbook of Qualitative Research , 1994 .

[7]  Janice M. Morse,et al.  Aggregating Qualitative Findings: An Approach to Theory Development , 1994 .

[8]  Deirdre A. Fullerton,et al.  Sexual health education interventions for young people: a methodological review , 1995, BMJ.

[9]  S. Oliver,et al.  Review of Effectiveness of Sexual Health Promotion Interventions for Young People (updated) , 1995 .

[10]  John W. Creswell,et al.  Research Design: Qualitative and Quantitative Approaches , 1997 .

[11]  Sally Sieloff Magnan,et al.  Research Design: Qualitative and Quantitative Approaches , 1997 .

[12]  A Harden,et al.  Effectiveness of health promotion interventions in the workplace: a review , 1998 .

[13]  S. Oliver,et al.  A Review of the Effectiveness of Workplace Health Promotion Interventions , 1998 .

[14]  Abbas Tashakkori,et al.  Mixed Methodology: Combining Qualitative and Quantitative Approaches , 1998 .

[15]  A Harden,et al.  A systematic review of the effectiveness of health promotion interventions in the workplace. , 1999, Occupational medicine.

[16]  N. Denzin,et al.  The discipline and practice of qualitative research [Introduction]. Denzin & Y. S. Lincoln (Eds.), The handbook of qualitative research (2nd ed. , 2000 .

[17]  Mark W. Lipsey,et al.  Practical Meta-Analysis , 2000 .

[18]  Sandy Oliver,et al.  Peer-delivered health promotion for young people: A systematic review of different study designs , 2001 .

[19]  Sandy Oliver,et al.  Young people and physical activity: a systematic review of research on barriers and facilitators , 2001 .

[20]  S. Oliver,et al.  Young people and mental health: a systematic review of research on barriers and facilitators , 2001 .

[21]  M. Hammersley.  On 'systematic' reviews of research literatures: A 'narrative' response to Evans & Benefield , 2001 .

[22]  Sally Thorne,et al.  Meta-Study of Qualitative Health Research , 2001 .

[23]  G. Vulliamy,et al.  The Social Construction of School Exclusion Rates: Implications for evaluation methodology , 2001 .

[24]  J. Elliott.  Making Evidence-based Practice Educational , 2001 .

[25]  Keith R Abrams,et al.  Factors affecting uptake of childhood immunisation: a Bayesian synthesis of qualitative and quantitative evidence , 2002, The Lancet.

[26]  Robert Willmott,et al.  Reclaiming Metaphysical Truth for Educational Research , 2002 .

[27]  Sandy Oliver,et al.  HIV health promotion and men who have sex with men (MSM): a systematic review of research relevant to the development and implementation of effective and appropriate interventions , 2004 .

[28]  Sandy Oliver,et al.  Children and physical activity: a systematic review of barriers and facilitators , 2003 .

[29]  Sandy Oliver,et al.  Children and healthy eating: a systematic review of barriers and facilitators , 2003 .

[30]  J. Sterne,et al.  How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. , 2003, Health technology assessment.

[31]  Practical systems for systematic reviews of research to inform policy and practice in education , 2003 .

[32]  Jenny Donovan,et al.  Evaluating meta-ethnography: a synthesis of qualitative research on lay experiences of diabetes and diabetes care. , 2003, Social science & medicine.

[33]  Alex J. Sutton,et al.  Integrative approaches to qualitative and quantitative evidence , 2004 .

[34]  Sandy Oliver,et al.  Applying systematic review methods to studies of people’s views: an example from public health research , 2004, Journal of Epidemiology and Community Health.

[35]  Sandy Oliver,et al.  Integrating qualitative research with trials in systematic reviews , 2004, BMJ : British Medical Journal.

[36]  Sandy Oliver,et al.  An Emerging Framework for Including Different Types of Evidence in Systematic Reviews for Public Policy , 2005 .

[37]  J Garcia,et al.  Young people and healthy eating: a systematic review of research on barriers and facilitators. , 2001, Health education research.