Meta-Analysis and Meta-Synthesis Methodologies: Rigorously Piecing Together Research

For a variety of reasons, education research can be difficult to summarize. Varying contexts, designs, levels of quality, measurement challenges, definition of underlying constructs, and treatments as well as the complexity of research subjects themselves can result in variability. Education research is voluminous and draws on multiple methods including quantitative, as well as, qualitative approaches to answer key research questions. With increased numbers of empirical research in Instructional Design and Technology (IDT), using various synthesis methods can provide a means to more deeply understand trends and patterns in research findings across multiple studies. The purpose of this article is to illustrate structured review or meta-synthesis procedures for qualitative research, as well as, novel meta-analysis procedures for the kinds of multiple treatment designs common to IDT settings. Sample analyses are used to discuss key methodological ideas as a way to introduce researchers to these techniques.

[1]  Anna Chaimani,et al.  Evaluating the Quality of Evidence from a Network Meta-Analysis , 2014, PloS one.

[2]  Daniel M. Maggin,et al.  Technology-Aided Instruction and Intervention for Students With ASD: A Meta-Analysis Using Novel Methods of Estimating Effect Sizes for Single-Case Research , 2017 .

[3]  Deborah Finfgeld-Connett,et al.  Generalizability and transferability of meta-synthesis research findings. , 2010, Journal of advanced nursing.

[4]  Andrew Thomas,et al.  The BUGS project: Evolution, critique and future directions , 2009, Statistics in medicine.

[5]  Sally Thorne,et al.  Meta-Study of Qualitative Health Research: A Practical Guide to Meta-Analysis and Meta-Synthesis , 2001 .

[6]  Beverly Park Woolf,et al.  Evaluation of AnimalWatch: An Intelligent Tutoring System for Arithmetic and Fractions , 2010 .

[7]  S. Downe,et al.  Meta-synthesis method for qualitative research: a literature review. , 2005, Journal of advanced nursing.

[8]  G. Noblit,et al.  Meta-Ethnography: Synthesizing Qualitative Studies , 1988 .

[9]  Rachelle S. Heller,et al.  The Role of Hypermedia in Education: A Look at the Research Issues. , 1990 .

[10]  L. Hedges,et al.  Introduction to Meta‐Analysis , 2009, International Coaching Psychology Review.

[11]  Susan Pedersen,et al.  Scaffolding middle school students’ content knowledge and ill-structured problem solving in a problem-based hypermedia learning environment , 2010 .

[12]  A. Walker,et al.  A Bayesian Network Meta-Analysis to Synthesize the Influence of Contexts of Scaffolding Use on Cognitive Outcomes in STEM Education , 2017, Review of educational research.

[13]  Jamaal Rashad Young,et al.  Technology-enhanced mathematics instruction: A second-order meta-analysis of 30 years of research , 2017 .

[14]  Susan McKenney,et al.  Conducting Educational Design Research , 2012 .

[15]  Mohammad Hassan Murad,et al.  A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis , 2014, BMJ : British Medical Journal.

[16]  Huifen Lin,et al.  A Meta-Synthesis of Empirical Research on the Effectiveness of Computer-Mediated Communication (CMC) in SLA. , 2015 .

[17]  R. Torraco.  Writing Integrative Literature Reviews , 2016 .

[18]  L. Fegran,et al.  Using Sandelowski and Barroso’s Meta-Synthesis Method in Advancing Qualitative Evidence , 2016, Qualitative health research.

[19]  D. Finfgeld-Connett.  Metasynthesis Findings , 2014, Qualitative health research.

[20]  James Thomas,et al.  Bmc Medical Research Methodology Methods for the Synthesis of Qualitative Research: a Critical Review , 2022 .

[21]  Matthew K. Burns,et al.  Updated Meta-Analysis of Learner Control Within Educational Technology , 2014 .

[22]  Brian R. Belland,et al.  Synthesizing Results From Empirical Research on Computer-Based Scaffolding in STEM Education , 2016, Review of educational research.

[23]  S. Mumtaz.  Factors affecting teachers' use of information and communications technology: a review of the literature , 2000 .

[24]  Margarete Sandelowski,et al.  Toward a metasynthesis of qualitative findings on motherhood in HIV-positive women. , 2003, Research in nursing & health.

[25]  K. Finlayson,et al.  Qualitative meta-synthesis: a guide for the novice. , 2008, Nurse researcher.

[26]  T. Reeves,et al.  The goals and methods of educational technology research over a quarter century (1989–2014) , 2017 .

[27]  D. Finfgeld,et al.  Metasynthesis: The State of the Art—So Far , 2003, Qualitative health research.

[28]  J. Higgins,et al.  Cochrane Handbook for Systematic Reviews of Interventions , 2010, International Coaching Psychology Review.

[29]  J. Sterne,et al.  The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials , 2011, BMJ : British Medical Journal.

[30]  Sally Thorne,et al.  Meta-Study of Qualitative Health Research , 2001 .

[31]  G. Glass.  Primary, Secondary, and Meta-Analysis of Research1 , 1976 .

[32]  Jessica Gurevitch,et al.  Meta-analysis and the science of research synthesis , 2018, Nature.

[33]  S. Higgins.  Meta‐synthesis and comparative meta‐analysis of education research findings: some risks and benefits , 2016 .

[34]  Deborah Finfgeld-Connett,et al.  The Future of Theory-Generating Meta-Synthesis Research , 2016, Qualitative health research.

[35]  Johan van Braak,et al.  Preparing pre-service teachers to integrate technology in education: A synthesis of qualitative evidence , 2012, Comput. Educ..

[36]  Jennifer Attride-Stirling.  Thematic networks: an analytic tool for qualitative research , 2001 .

[37]  F. Stiefel,et al.  Multicomponent Intervention for Patients Admitted to an Emergency Unit for Suicide Attempt: An Exploratory Study , 2017, Front. Psychiatry.

[38]  M. Moro,et al.  Metasynthesis: An Original Method to Synthesize Qualitative Literature in Psychiatry , 2017, Front. Psychiatry.

[39]  M. Sandelowski,et al.  Qualitative Metasynthesis: Reflections on Methodological Orientation and Ideological Agenda , 2004, Qualitative health research.

[40]  M Sandelowski,et al.  Qualitative Metasynthesis : Issues and Techniques , 1997 .

[41]  N. Bhatnagar,et al.  Multiple treatment and indirect treatment comparisons: An overview of network meta-analysis , 2014, Perspectives in clinical research.

[42]  T. Lumley.  Network meta‐analysis for indirect treatment comparisons , 2002, Statistics in medicine.

[43]  A. Owen,et al.  Combined resynchronisation and implantable defibrillator therapy in left ventricular dysfunction: Bayesian network meta-analysis of randomised controlled trials , 2007, BMJ : British Medical Journal.

[44]  Jean Marie Tuquero,et al.  A Meta-ethnographic Synthesis of Support Services in Distance Learning Programs , 2011, J. Inf. Technol. Educ. Innov. Pract..

[45]  Ying Liu,et al.  ICT and Special Educational Needs: Using Meta-synthesis for Bridging the Multifaceted Divide , 2007, International Conference on Computational Science.

[46]  Penny M. Beile,et al.  Scholars Before Researchers: On the Centrality of the Dissertation Literature Review in Research Preparation , 2005 .