The effectiveness of intelligent tutoring systems on K-12 students' reading comprehension: A meta-analysis

This meta‐analysis examined the effectiveness of improving reading comprehension for students in K‐12 classrooms using intelligent tutoring systems (ITSs), a computer‐based learning environment that provides customizable and immediate feedback to the learner. Nineteen studies from 13 publications incorporating approximately 10 000 students were included in the final analysis; using robust variance estimation to account for statistical dependencies, the 19 studies yielded 88 effect size estimates. The meta‐analysis indicated that the overall random effect size of ITSs on reading comprehension was 0.60 (using a mix of standardized and researcher‐designed measures) with a 95% confidence interval 0.36 to 0.85 (p < 0.001). This review confirms previous studies comparing ITSs to human tutoring: ITSs produced a small effect size when compared to human tutoring (0.20, 0.02–0.38, p = 0.036, n = 21). All comparisons to human tutoring used standardized measures. This review also found that ITSs produced a larger effect size on reading comprehension when compared to traditional instruction (0.86) for mixed measures and (0.26) for standardized measures. These findings may be of interest to practitioners and policy makers seeking to improve reading comprehension using consistent and accessible ITSs. Recommendations for researchers include conducting studies to understand the difference between traditional and updated versions of ITSs and employing valid and reliable standardized tests and researcher‐designed measures. [ABSTRACT FROM AUTHOR]

[1]  Jack Mostow,et al.  Computer-Guided Oral Reading versus Independent Practice: Comparison of Sustained Silent Reading to an Automated Reading Tutor That Listens , 2013 .

[2]  Valerie J. Shute,et al.  Using an Evidence-Based Approach to Assess Mental Models , 2008 .

[3]  Chen-Lin C. Kulik,et al.  Educational Outcomes of Tutoring: A Meta-analysis of Findings , 1982 .

[4]  Claire M. Fletcher-Flinn,et al.  The Efficacy of Computer Assisted Instruction (CAI): A Meta-Analysis , 1995 .

[5]  Bonnie J. F. Meyer,et al.  Multisite Randomized Controlled Trial Examining Intelligent Tutoring of Structure Strategy for Fifth-Grade Readers , 2014 .

[6]  H. Cooper,et al.  A meta-analysis of the effectiveness of intelligent tutoring systems on K–12 students’ mathematical learning. , 2013 .

[7]  Kurt VanLehn,et al.  The Behavior of Tutoring Systems , 2006, Int. J. Artif. Intell. Educ..

[8]  R C Atkinson,et al.  Computerized instruction and the learning process. , 1968, The American psychologist.

[9]  Danielle S. McNamara,et al.  The impact of individual differences on learning with an educational game and a traditional ITS , 2013, Int. J. Learn. Technol..

[10]  Robert E. Slavin,et al.  Effects of Educational Technology Applications on Reading Outcomes for Struggling Readers: A Best Evidence Synthesis. , 2012 .

[11]  Danielle S. McNamara,et al.  Improving Adolescent Students' Reading Comprehension with Istart , 2006 .

[12]  B. Bloom.  The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring , 1984 .

[13]  Zhihong Xu,et al.  The effectiveness of educational technology applications on adult English language learners’ writing quality: a meta-analysis , 2018, Computer Assisted Language Learning.

[14]  Elizabeth Tipton,et al.  Robust variance estimation with dependent effect sizes: practical considerations including a software tutorial in Stata and spss , 2014, Research synthesis methods.

[15]  S. Walpole,et al.  An analysis of the ecological components within a text structure intervention , 2018 .

[16]  Eduardo Vidal-Abarca,et al.  TuinLEC, an intelligent tutoring system to improve reading literacy skills / TuinLEC, un tutor inteligente para mejorar la competencia lectora , 2014 .

[17]  Philip C. Abrami,et al.  Technology Infusion in Success for All: Reading Outcomes for First Graders , 2008, The Elementary School Journal.

[18]  Robert E. Slavin,et al.  How features of educational technology applications affect student reading outcomes: A meta-analysis , 2012 .

[19]  Bonnie J. F. Meyer,et al.  Web-Based Tutoring of the Structure Strategy With or Without Elaborated Feedback or Choice for Fifth- and Seventh-Grade Readers , 2010 .

[20]  Olusola O. Adesope,et al.  Intelligent tutoring systems and learning outcomes: A meta-analysis , 2014 .

[21]  T. Shanahan,et al.  Reading recovery: An independent evaluation of the effects of an early instructional intervention for at-risk learners. , 1995 .

[22]  Erin K. Washburn,et al.  Peter Effect in the Preparation of Reading Teachers , 2012 .

[23]  H. Cooper,et al.  A meta-analysis of the effectiveness of intelligent tutoring systems on college students' academic learning , 2014 .

[24]  Bonnie J. F. Meyer,et al.  Large-scale randomized controlled trial with 4th graders using intelligent tutoring of the structure strategy to improve nonfiction reading comprehension , 2012 .

[25]  K. VanLehn.  The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems, and Other Tutoring Systems , 2011 .

[26]  Jessica A. Brown,et al.  The Effects of Text Structure Instruction on Expository Reading Comprehension: A Meta-Analysis , 2016 .

[27]  James A. Kulik,et al.  Effectiveness of Intelligent Tutoring Systems , 2016 .

[28]  M. E. Otter,et al.  Computer-Assisted Instruction in Support of Beginning Reading Instruction: A Review , 2002 .

[29]  Peter Grogono,et al.  Modelling techniques for tutoring systems , 1993 .

[30]  Philip C. Abrami,et al.  Computer-Assisted Tutoring in Success for All: Reading Outcomes for First Graders , 2008 .

[31]  Pipatsarun Phobun,et al.  Adaptive intelligent tutoring systems for e-learning systems , 2010 .

[32]  Philip C. Abrami,et al.  Small-Group, Computer-Assisted Tutoring to Improve Reading Outcomes for Struggling First and Second Graders , 2011, The Elementary School Journal.

[33]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[34]  Scott Dooley,et al.  Summary Street®: Computer Support for Comprehension and Writing , 2005 .

[35]  Thomas K. Landauer,et al.  On the computational basis of learning and cognition: Arguments from LSA , 2002 .

[36]  Bonnie J. F. Meyer,et al.  High-fidelity implementation of web-based intelligent tutoring system improves fourth and fifth graders content area reading comprehension , 2013, Comput. Educ..

[37]  John R. Anderson,et al.  Cognitive Tutors: Lessons Learned , 1995 .

[38]  Alysia D. Roehrig,et al.  The First Days of School in the Classrooms of Two More Effective and Four Less Effective Primary-Grades Teachers , 2004, The Elementary School Journal.

[39]  Elizabeth Tipton,et al.  Robust variance estimation in meta‐regression with dependent effect size estimates , 2010, Research synthesis methods.

[40]  Herbert Baier,et al.  Rethinking Strategy Instruction: Direct Reading Strategy Instruction versus Computer-Based Guided Practice , 2013 .

[41]  Bonnie J. F. Meyer,et al.  Web-Based Text Structure Strategy Instruction Improves Seventh Graders’ Content Area Reading Comprehension , 2017 .

[42]  B. Meyer,et al.  Effects of structure strategy instruction delivered to fifth-grade children using the Internet with and without the aid of older adult tutors. , 2002 .