Advancements in Research Synthesis Methods: From a Methodologically Inclusive Perspective

The dominant literature on research synthesis methods has positivist and neo-positivist origins. In recent years, the landscape of research synthesis methods has changed rapidly to become inclusive. This article highlights methodologically inclusive advancements in research synthesis methods. Attention is drawn to insights from interpretive, critical, and participatory traditions for enhancing trustworthiness, utility, and/or emancipatory potential for research syntheses. Also noted is a paucity of the literature that builds connections between methodologically diverse segments of the literature on research synthesis methods. Salient features of a methodologically inclusive research synthesis (MIRS) framework are described. The MIRS framework has been conceptualized by distilling and synthesizing ideas, theories, and strategies from the extensive literatures on research synthesis methods and primary research methods. Rather than prescribe how a research synthesis should be conducted or evaluated, this article attempts to open spaces, raise questions, explore possibilities, and contest taken-for-granted practices.

[1]  Paul D. Williams,et al.  Should All Literature Reviews be Systematic? , 2000 .

[2]  H. Cooper Synthesizing Research: A Guide for Literature Reviews , 1998 .

[3]  R. Coe,et al.  Evidence and Indicators: Dialogue, Improvement and Researching for Others , 2000 .

[4]  T. Greenhalgh,et al.  Realist review - a new method of systematic review designed for complex policy interventions , 2005, Journal of health services research & policy.

[5]  P. Davies The Relevance of Systematic Reviews to Educational Policy and Practice , 2000 .

[6]  Rodney T. Ogawa,et al.  A Response to Commentaries on “Towards Rigor in Reviews of Multivocal Literatures…” , 1991 .

[7]  S. Downe,et al.  Meta-synthesis method for qualitative research: a literature review. , 2005, Journal of advanced nursing.

[8]  Threads of a New Language: A Response to Eisenhart’s ‘On the Subject of Interpretive Review’ , 1998 .

[9]  G. Glass,et al.  Meta-analysis in social research , 1981 .

[10]  Patti Lather Research as Praxis , 1986 .

[11]  J. Popay,et al.  Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field , 2005, Journal of health services research & policy.

[12]  Lela V. Zimmer,et al.  Qualitative meta-synthesis: a question of dialoguing with texts. , 2006, Journal of advanced nursing.

[13]  B. M. Franklin,et al.  Discourse, Rationality, and Educational Research: A Historical Perspective of RER 1 , 1999 .

[14]  N. Denzin,et al.  Handbook of Qualitative Research , 1994 .

[15]  Shanyang Zhao Metatheory, Metamethod, Meta-Data-Analysis: What, Why, and How? , 1991 .

[16]  Kenneth A. Strike,et al.  Types of synthesis and their criteria , 1983 .

[17]  M. Eisenhart,et al.  On the Subject of Interpretive Reviews1 , 1998 .

[18]  Richard Andrews,et al.  THE PLACE OF SYSTEMATIC REVIEWS IN EDUCATION RESEARCH , 2005 .

[19]  Richard V. R. Thomas,et al.  Evidence-based Practice in Education , 2004 .

[20]  J. A. Smith,et al.  The problem of appraising qualitative research , 2004, Quality and Safety in Health Care.

[21]  Harsh Suri,et al.  Revisiting Methods of Literature Synthesis. , 1999 .

[22]  Seeing the forest for the trees: a qualitative synthesis project. , 2003, The Journal of nursing education.

[23]  W. Whyte,et al.  Participatory Action Research , 1989 .

[24]  M. Sandelowski,et al.  Reading Qualitative Studies , 2002 .

[25]  Penny M. Beile,et al.  Scholars Before Researchers: On the Centrality of the Dissertation Literature Review in Research Preparation , 2005 .

[26]  Carol E. Kasworm Adult Undergraduates in Higher Education: A Review of Past Research Perspectives , 1990 .

[27]  Patti Lather Issues of validity in openly ideological research: Between a rock and a soft place , 1986 .

[28]  Patti Lather FERTILE OBSESSION: Validity After Poststructuralism , 1993 .

[29]  F. Baum,et al.  Participatory action research , 2006, Journal of Epidemiology and Community Health.

[30]  Einar M. Rønquist,et al.  Foreword , 1999, Biological Psychiatry.

[31]  G. Glass Primary, Secondary, and Meta-Analysis of Research1 , 1976 .

[32]  A. Peshkin Angles of Vision: Enhancing Perception in Qualitative Research , 2001 .

[33]  Robert E. Slavin,et al.  Best-Evidence Synthesis: An Alternative to Meta-Analytic and Traditional Reviews , 1986 .

[34]  Sue Dockett,et al.  Australian association for research in education , 1989 .

[35]  Harris Cooper,et al.  Effect Size Substantive Interpretation Guidelines : Issues in the Interpretation of Effect Sizes , 2007 .

[36]  Carl A. Grant,et al.  (Re)Viewing a Review: A Case History of the Review of Educational Research , 1999 .

[37]  Bruce Thompson,et al.  Research Synthesis: Effect Sizes , 2006 .

[38]  H. Cooper Organizing knowledge syntheses: A taxonomy of literature reviews , 1988 .

[39]  David B. Pillemer,et al.  Summing Up: The Science of Reviewing Research , 1984 .

[40]  D. Finfgeld,et al.  Metasynthesis: The State of the Art—So Far , 2003, Qualitative health research.

[41]  L. Richardson Getting personal: Writing-stories , 2001 .

[42]  Margarete Sandelowski,et al.  Creating Metasummaries of Qualitative Findings , 2003, Nursing research.

[43]  F. Murray,et al.  Factors in the Peer Review of Reviews , 1996 .

[44]  M. Strathern The Tyranny of Transparency , 2000 .

[45]  L. Jensen,et al.  A Synthesis of Qualitative Research on Wellness-Illness , 1994 .

[46]  Lyn Yates,et al.  Research methodology, education, and theoretical fashions: Constructing a methodology course in an era of deconstruction , 1997 .

[47]  Janice M. Morse,et al.  Aggregating Qualitative Findings: An Approach to Theory Development , 1994 .

[48]  William L. Curlette,et al.  A meta-synthesis of factors related to educational achievement: a methodological approach to summarizing and synthesizing meta-analyses , 1996 .

[49]  Gene V. Glass,et al.  Meta-Analysis: The Quantitative Synthesis of Research Findings , 2006 .

[50]  R. Yin Advancing Rigorous Methodologies: A Review of “Towards Rigor in Reviews of Multivocal Literatures… ” , 1991 .

[51]  Sally Thorne,et al.  Meta-Study of Qualitative Health Research: A Practical Guide to Meta-Analysis and Meta-Synthesis , 2001 .

[52]  N. Denzin Aesthetics and the Practices of Qualitative Inquiry , 2000 .

[53]  Michael Jackson,et al.  Minima Ethnographica: Intersubjectivity and the Anthropological Project , 1998 .

[54]  T. Kuhn The structure of scientific revolutions, 3rd ed. , 1996 .

[55]  R. Whittemore,et al.  Combining Evidence in Nursing Research: Methods and Implications , 2005, Nursing research.

[56]  S. Ward,et al.  Knowledge structure and use : implications for synthesis and interpretation , 1983 .

[57]  What Kind of Evidence does Government Need? , 2004 .

[58]  Patti Lather,et al.  To Be of Use: The Work of Reviewing , 1999 .

[59]  M. Hammersley On 'systematic' reviews of research literatures: A 'narrative' response to Evans & Benefield , 2001 .

[60]  J. Gee It's Theories All the Way Down: A Response to Scientific Research in Education. , 2005 .

[61]  Alex J. Sutton,et al.  Integrative approaches to qualitative and quantitative evidence , 2004 .

[62]  Margarete Sandelowski,et al.  Finding the findings in qualitative studies. , 2002, Journal of nursing scholarship : an official publication of Sigma Theta Tau International Honor Society of Nursing.

[63]  Thomas A. Schwandt A Diagnostic Reading of Scientifically Based Research for Education. , 2005 .

[64]  Selcuk R. Sirin Socioeconomic Status and Academic Achievement: A Meta-Analytic Review of Research , 2005 .

[65]  S. McDonald,et al.  The Campbell Collaboration Social, Psychological, Educational and Criminological Trials Register (C2-SPECTR) to Facilitate the Preparation and Maintenance of Systematic Reviews of Social and Educational Interventions , 2000 .

[66]  D. Caulley Qualitative research for education: An introduction to theories and methods , 2007 .

[67]  D. Anthony Evidence-based Policy: A Realist Perspective , 2007 .

[68]  L. Hedges,et al.  The Handbook of Research Synthesis , 1995 .

[69]  B. Baker What is Voice? Issues of Identity and Representation in the Framing of Reviews , 1999 .

[70]  T. Kuhn,et al.  The Structure of Scientific Revolutions , 1963 .

[71]  R. Hanka The Handbook of Research Synthesis , 1994 .

[72]  Robert K. Yin,et al.  The Difference That Quality Makes , 1976 .

[73]  Ray Pawson,et al.  Evidence-based Policy: The Promise of `Realist Synthesis' , 2002 .

[74]  Jolie A. Mayer-Smith,et al.  A Critical Analysis of the Research on Learning to Teach: Making the Case for an Ecological Perspective on Inquiry , 1998 .

[75]  R. Light Synthesis methods: Some judgement calls that must be made , 1980 .

[76]  R. Slavin Evidence-Based Education Policies: Transforming Educational Practice and Research , 2002 .

[77]  Guarding the Castle and Opening the Gates , 2004 .

[78]  P. Sleight,et al.  Publication bias , 1991, The Lancet.

[79]  Louise A. Jensen,et al.  Meta-Synthesis of Qualitative Findings , 1996 .

[80]  M. Apple What Counts as Legitimate Knowledge? The Social Production and Use of Reviews , 1999 .

[81]  Stanley Aronowitz,et al.  Postmodern Education: Politics, Culture, and Social Criticism , 1991 .

[82]  H. Suri Evidence-based education: From an epistemologically inclusive perspective , 2007 .

[83]  M. Dunkin Types of Errors in Synthesizing Research in Education , 1996 .

[84]  P. Moss UNDERSTANDING THE OTHER/UNDERSTANDING OURSELVES: TOWARD A CONSTRUCTIVE DIALOGUE ABOUT “PRINCIPLES’ IN EDUCATIONAL RESEARCH , 2005 .

[85]  Daniel L. Schwartz,et al.  Rethinking transfer: A simple proposal with multiple implica-tions , 1999 .

[86]  H. Suri Ethical considerations in synthesising research ‐ Whose representations? , 2008 .

[87]  Laura C. Leviton,et al.  Reviewing the literature: A comparison of traditional methods with meta-analysis. , 1980 .

[88]  Michael Quinn Patton,et al.  Towards Utility in Reviews of Multivocal Literatures , 1991 .

[89]  David Gough,et al.  Systematic research synthesis , 2004 .

[90]  Mark E. Engberg Improving Intergroup Relations in Higher Education: A Critical Examination of the Influence of Educational Interventions on Racial Bias , 2004 .

[91]  K. Howe THE QUESTION OF EDUCATION SCIENCE: EXPERIMENTISM VERSUS EXPERIMENTALISM , 2005 .

[92]  M. Petticrew,et al.  Systematic Reviews in the Social Sciences: A Practical Guide , 2005 .

[93]  George W. Noblit,et al.  Meta-Ethnography: Issues in the Synthesis and Replication of Qualitative Research. , 1983 .

[94]  S. Clegg Evidence‐based practice in educational research: a critical realist critique of systematic review , 2005 .

[95]  W. Harlen,et al.  Opportunities and Challenges of Using Systematic Reviews of Research for Evidence-based Policy in Education , 2004 .

[96]  T. Popkewitz Reviewing Reviews: RER, Research, and the Politics of Educational Knowledge , 1999 .

[97]  Diane Crawford,et al.  Editorial , 2000, CACM.

[98]  D. Sackett,et al.  Cochrane Collaboration , 1994, BMJ.

[99]  G. Noblit,et al.  Meta-Ethnography: Synthesizing Qualitative Studies , 1988 .

[100]  Marvin Lazerson,et al.  New Directions in American Educational History: Problems and Prospects , 2000 .

[101]  Rodney T. Ogawa,et al.  Towards Rigor in Reviews of Multivocal Literatures: Applying the Exploratory Case Study Method , 1991 .

[102]  Mary Dixon-Woods,et al.  Synthesising qualitative and quantitative evidence: a review of possible methods. , 2005, Journal of health services research & policy.

[103]  Yvonna S. Lincoln,et al.  Naturalistic and Rationalistic Enquiry , 1999 .

[104]  Colin Sharp Qualitative Research and Evaluation Methods (3rd ed.) , 2003 .

[105]  Diane M. Tomasic,et al.  Meta-Analysis: A Comparison of Approaches , 2005 .

[106]  Grace Livingston Beyond Watching Over Established Ways: A Review as Recasting the Literature, Recasting the Lived , 1999 .

[107]  C. Gipps,et al.  Watching the Watchers: research, methods, politics and equity. A Response to Foster & Hammersley , 1998 .

[108]  Ann Oakley,et al.  Research evidence, knowledge management and educational practice: early lessons from a systematic approach , 2003 .

[109]  M Sandelowski,et al.  Focus on qualitative methods. Qualitative metasynthesis: issues and techniques. , 1997, Research in nursing & health.

[110]  M. Hammersley Some questions about evidence-based practice in education , 2001 .

[111]  Daniel L. Schwartz,et al.  Chapter 3: Rethinking Transfer: A Simple Proposal With Multiple Implications , 1999 .

[112]  C. Teddlie,et al.  SAGE Handbook of Mixed Methods in Social & Behavioral Research , 2010 .

[113]  Thomas A. Schwandt,et al.  Judging interpretations: But is it rigorous? trustworthiness and authenticity in naturalistic evaluation , 2007 .

[114]  Harsh Suri Essential Features of Methodologically Inclusive Research Syntheses. , 2002 .

[115]  Jennifer Grant Haworth,et al.  Doctoral Student Attrition and Persistence: A Meta-Synthesis of Research. ASHE Annual Meeting Paper. , 1999 .

[116]  Harris Cooper,et al.  Scientific Guidelines for Conducting Integrative Research Reviews , 1982 .

[117]  Sally Thorne,et al.  Meta-Study of Qualitative Health Research , 2001 .

[118]  Torsten Husén Research paradigms in education , 1988 .

[119]  Mark W. Lipsey,et al.  Practical Meta-Analysis , 2000 .

[120]  M. Hammersley,et al.  A Review of Reviews: structure and function in reviews of educational research , 1998 .

[121]  Paul M. Wortman,et al.  Judging research quality. , 1994 .

[122]  J. Popay,et al.  Synthesizing Qualitative and Quantitative Health Evidence: A Guide to Methods , 2007 .

[123]  Thomas D. Cook,et al.  Threats to the validity of research syntheses , 1994 .

[124]  S. Gorard The Role of Causal Models in Evidence-informed Policy Making and Practice , 2002 .

[125]  William Shadish David Myers Campbell Collaboration Research Design Policy Brief , 2004 .

[126]  M. Dixon-Woods,et al.  The problem of appraising qualitative research. , 2004 .

[127]  M. MacLure,et al.  ‘Clarity bordering on stupidity’: where’s the quality in systematic review? , 2005 .

[128]  M. Patton Qualitative research and evaluation methods , 1980 .

[129]  M. Windschitl Framing Constructivism in Practice as the Negotiation of Dilemmas: An Analysis of the Conceptual, Pedagogical, Cultural, and Political Challenges Facing Teachers , 2002 .

[130]  P. Reason,et al.  A Participatory Inquiry Paradigm , 1997 .

[131]  David R. Jones,et al.  How can systematic reviews incorporate qualitative research? A critical perspective , 2006 .

[132]  J. Popay,et al.  Rationale and Standards for the Systematic Review of Qualitative Literature in Health Services Research , 1998, Qualitative health research.

[133]  Margarete Sandelowski,et al.  Classifying the Findings in Qualitative Studies , 2003, Qualitative health research.

[134]  David B. Pillemer,et al.  Numbers and Narrative: Combining Their Strengths in Research Reviews , 1982 .