Does the Flipped Classroom Improve Student Learning and Satisfaction? A Systematic Review and Meta-Analysis

We searched and meta-analyzed studies comparing flipped classroom teaching with traditional, lecture-based teaching to evaluate the evidence for the flipped classroom’s influence on continuous-learning measures, pass/fail rates, and student evaluations of teaching. Eight electronic reference databases were searched to retrieve relevant studies. Our results indicate a small effect in favor of the flipped classroom on learning (Hedges’ g = 0.35, 95% confidence interval [CI] [0.31, 0.40], k = 272). However, analyses restricted to studies with sufficient power resulted in an estimate of 0.24 (95% CI [0.18, 0.31], k = 90). Effects on pass rates (odds ratio = 1.55, 95% CI [1.34, 1.78], k = 45) and student satisfaction (Hedges’ g = 0.16, 95% CI [0.06, 0.26], k = 69) were small and also likely influenced by publication bias. There is some support for the notion that the positive impact on learning may increase slightly if testing student preparation is part of the implementation.

[1]  L. Hedges,et al.  Fixed- and random-effects models in meta-analysis. , 1998 .

[2]  A. Greenwald Consequences of Prejudice Against the Null Hypothesis , 1975 .

[3]  Jacob Cohen,et al.  The Equivalence of Weighted Kappa and the Intraclass Correlation Coefficient as Measures of Reliability , 1973 .

[4]  C. Mellis,et al.  Applying Established Guidelines to Team-Based Learning Programs in Medical Schools: A Systematic Review , 2014, Academic medicine : journal of the Association of American Medical Colleges.

[5]  Alex J Sutton,et al.  Performance of the trim and fill method in the presence of publication bias and between‐study heterogeneity , 2007, Statistics in medicine.

[6]  Carly D. Robinson,et al.  Taking It to the Next Level: A Field Experiment to Improve Instructor-Student Relationships in College , 2019, AERA Open.

[7]  A Alexander Beaujean,et al.  The Effectiveness of Team-Based Learning on Academic Outcomes: A Meta-Analysis , 2017 .

[8]  William E. Cashin,et al.  Student Ratings of Teaching : A Summary of Research and Literature , 2012 .

[9]  J. Ioannidis Why Most Discovered True Associations Are Inflated , 2008, Epidemiology.

[10]  Jeffrey D. Karpicke Retrieval-Based Learning , 2012 .

[11]  C. Begg,et al.  Operating characteristics of a rank correlation test for publication bias. , 1994, Biometrics.

[12]  Matthew J. Mayhew,et al.  Ten Challenges and Recommendations for Advancing Research on the Effects of College on Students , 2017 .

[13]  L. Fink,et al.  Team-based learning : a transformative use of small groups , 2002 .

[14]  Murat Akçayir,et al.  The flipped classroom: A review of its advantages and challenges , 2018, Comput. Educ..

[15]  John E. Hunter,et al.  Fixed Effects vs. Random Effects Meta‐Analysis Models: Implications for Cumulative Research Knowledge , 2000 .

[16]  Cui Tan,et al.  Effectiveness of flipped classrooms in nursing education: Systematic review and meta-analysis , 2017 .

[17]  A. Field Meta-analysis of correlation coefficients: a Monte Carlo comparison of fixed- and random-effects methods. , 2001, Psychological methods.

[18]  John Dunlosky,et al.  Improving Students’ Learning With Effective Learning Techniques: Promising Directions From Cognitive and Educational Psychology , 2012 .

[19]  Nancy A. Schiller,et al.  A Chat with the Survey Monkey: Case Studies and the Flipped Classroom , 2014 .

[20]  J. Sterne,et al.  Publication and related bias in meta-analysis: power of statistical tests and prevalence in the literature. , 2000, Journal of clinical epidemiology.

[21]  Lisa S. Smith,et al.  A Meta-Analysis of Outcomes Comparing Flipped Classroom and Lecture , 2018, American Journal of Pharmaceutical Education.

[22]  J. Higgins,et al.  Cochrane Handbook for Systematic Reviews of Interventions, Version 5.1.0. The Cochrane Collaboration , 2013 .

[23]  P. Lachenbruch Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[24]  Jacob Bishop,et al.  A Controlled Study of the Flipped Classroom with Numerical Methods for Engineers. , 2013 .

[25]  J. Tagg,et al.  From Teaching to Learning — A New Paradigm For Undergraduate Education , 1995 .

[26]  Efrain Mercado,et al.  How ASCD Supports the Common Core. , 2013 .

[27]  M. McDaniel,et al.  Test-enhanced learning in the classroom: long-term improvements from quizzing. , 2011, Journal of experimental psychology. Applied.

[28]  Charles T. Jahren,et al.  A systematic review of research on the flipped learning method in engineering education , 2018, Br. J. Educ. Technol..

[29]  C. D. Ratta,et al.  Flipping the classroom with team-based learning in undergraduate nursing education. , 2015, Nurse educator.

[30]  P. Sleight,et al.  Publication bias , 1991, The Lancet.

[31]  Jan Keith Farmer,et al.  Where ' s the evidence that active learning works ? , 2006 .

[32]  D. Moher,et al.  CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials , 2010, Journal of clinical epidemiology.

[33]  Betty Love,et al.  Student learning and perceptions in a flipped linear algebra course , 2014 .

[34]  Robin Brewer,et al.  Successful stories and conflicts: A literature review on the effectiveness of flipped learning in higher education , 2018, J. Comput. Assist. Learn..

[35]  H. Goldstein,et al.  A New Framework for Systematic Reviews , 2014 .

[36]  J. Valentine,et al.  Outcome-Reporting Bias in Education Research , 2013 .

[37]  Wolfgang Stroebe,et al.  Why Good Teaching Evaluations May Reward Bad Teaching , 2016, Perspectives on psychological science : a journal of the Association for Psychological Science.

[38]  P. Spooren,et al.  On the Validity of Student Evaluation of Teaching , 2013 .

[39]  J. Hattie Which Strategies Best Enhance Teaching and Learning in Higher Education , 2011 .

[40]  Kenneth A. Bruffee,et al.  The Art of Collaborative Learning , 1987 .

[41]  Hsiu-Ting Hung,et al.  Flipping the classroom for English language learners to foster active learning , 2015 .

[42]  Michelle K. Smith,et al.  Active learning increases student performance in science, engineering, and mathematics , 2014, Proceedings of the National Academy of Sciences.

[43]  R. Tweedie,et al.  A Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis , 2000 .

[44]  L. Abeysekera,et al.  Motivation and cognitive load in the flipped classroom: definition, rationale and a call for research , 2015 .

[45]  Nancy Schoofs,et al.  Flipping the Classroom in Health Care Higher Education: A Systematic Review , 2019, Nurse educator.

[46]  Albert D. Ritzhaupt,et al.  Effects of the flipped classroom instructional strategy on students’ learning outcomes: a meta-analysis , 2018, Educational Technology Research and Development.

[47]  Yu-Che Chang,et al.  Academic outcomes of flipped classroom learning: a meta‐analysis , 2018, Medical education.

[48]  Anthony G. Greenwald,et al.  No pain, no gain? The importance of measuring course workload in student ratings of instruction , 1997 .

[49]  Frank A. Fear,et al.  Meaning Making and “The Learning Paradigm”: A Provocative Idea in Practice , 2003 .

[50]  Njål Foldnes The flipped classroom and cooperative learning: Evidence from a randomised experiment , 2016 .

[51]  Sarah L. Eddy,et al.  The ICAP Active Learning Framework Predicts the Learning Gains Observed in Intensely Active Classroom Experiences , 2017 .

[52]  P. Stark,et al.  Student Evaluations of Teaching (Mostly) Do Not Measure Teaching Effectiveness , 2016 .

[53]  David B. Pillemer,et al.  Summing Up: The Science of Reviewing Research , 1984 .

[54]  Lynn Summers,et al.  Flipping the Classroom to Improve Student Performance and Satisfaction , 2022 .

[55]  Khe Foon Hew,et al.  Flipped classroom improves student learning in health professions education: a meta-analysis , 2018, BMC Medical Education.

[56]  B. Zimmerman Becoming a Self-Regulated Learner: Which Are the Key Subprocesses?. , 1986 .

[57]  Xiaolian Jiang,et al.  Effectiveness of flipped classrooms in Chinese baccalaureate nursing education: A meta-analysis of randomized controlled trials. , 2018, International journal of nursing studies.

[58]  Pamela A. Moss,et al.  Standards for Reporting on Empirical Social Science Research in AERA Publications American Educational Research Association , 2006 .

[59]  John A. Centra,et al.  Will Teachers Receive Higher Student Evaluations by Giving Higher Grades and Less Course Work? , 2003 .

[60]  Embry-Riddle Aeronautical,et al.  The Flipped Classroom: A Survey of the Research , 2013 .

[61]  S Duval,et al.  Trim and Fill: A Simple Funnel‐Plot–Based Method of Testing and Adjusting for Publication Bias in Meta‐Analysis , 2000, Biometrics.

[62]  Mona Lundin,et al.  Higher education dominance and siloed knowledge: a systematic review of flipped classroom research , 2018, International Journal of Educational Technology in Higher Education.

[63]  C. Phillips,et al.  The use of flipped classrooms in higher education: A scoping review , 2015, Internet High. Educ..

[64]  David W. Johnson,et al.  Learning Together and Alone. Cooperative, Competitive, and Individualistic Learning. Fourth Edition. , 1991 .

[65]  Kay Dickersin,et al.  Publication Bias: Recognizing the Problem, Understanding Its Origins and Scope, and Preventing Harm , 2006 .

[66]  Scott Freeman,et al.  Reply to Hora: Meta-analytic techniques are designed to accommodate variation in implementation , 2014, Proceedings of the National Academy of Sciences.

[67]  สุธััญญา ด้วงอินทร์,et al.  Flip your classroom: Reach every student in every class every day , 2017 .

[68]  Michael J. Prince,et al.  Does Active Learning Work? A Review of the Research , 2004 .

[69]  John Hattie,et al.  Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement , 2008 .

[70]  R. Rosenthal The file drawer problem and tolerance for null results , 1979 .

[71]  J. Michael Where's the evidence that active learning works? , 2006, Advances in physiology education.

[72]  J. Hattie The applicability of Visible Learning to higher education. , 2015 .

[73]  Susan M. Martinelli,et al.  A systematic review of the effectiveness of flipped classrooms in medical education , 2017, Medical education.

[74]  David W. Johnson,et al.  Cooperative Learning: Improving University Instruction by Basing Practice on Validated Theory. , 2014 .

[75]  Evan Mayo-Wilson,et al.  Journal article reporting standards for quantitative research in psychology: The APA Publications and Communications Board task force report. , 2018, The American psychologist.

[76]  Jacob Cohen Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[77]  Rob Foels,et al.  Meta-Analysis of Group Learning Activities: Empirically Based Teaching Recommendations , 2012 .

[78]  R. Kornhaber,et al.  The evidence for 'flipping out': A systematic review of the flipped classroom in nursing education. , 2016, Nurse education today.

[79]  Mark W. Lipsey,et al.  Translating the Statistical Representation of the Effects of Education Interventions Into More Readily Interpretable Forms , 2012 .

[80]  Daniela Wong Gonzalez,et al.  Meta-analysis of faculty's teaching effectiveness: Student evaluation of teaching ratings and student learning are not related , 2017 .

[81]  A. Field Is the meta-analysis of correlation coefficients accurate when population correlations vary? , 2005, Psychological methods.

[82]  Brian A. Nosek,et al.  Power failure: why small sample size undermines the reliability of neuroscience , 2013, Nature Reviews Neuroscience.

[83]  John Dunlosky,et al.  TEACHER-READY RESEARCH REVIEW Practice Tests, Spaced Practice, and Successive Relearning: Tips for Classroom Use and for Guiding Students' Learning , 2015 .

[84]  L. Hedges,et al.  Introduction to Meta‐Analysis , 2009, International Coaching Psychology Review.

[85]  Ingram Olkin,et al.  Estimation of a Single Effect Size: Parametric and Nonparametric Methods , 1985 .

[86]  Herbert W. Marsh,et al.  Distinguishing Between Good (Useful) and Bad Workloads on Students’ Evaluations of Teaching , 2001 .

[87]  B. Zimmerman,et al.  Handbook of Self-Regulation of Learning and Performance , 2011 .

[88]  David A. Sears,et al.  Effects of Small-Group Learning on Transfer: a Meta-Analysis , 2015 .