Implementing an adaptive intelligent tutoring system as an instructional supplement

Evidence is emerging that technology-based curricula and adaptive learning systems can personalize students' learning experiences and facilitate development of mathematical skills. Yet, evidence of efficacy in rigorous studies for these blended instructional models is mixed. These studies highlight challenges implementing the systems in classrooms, which may contribute to a lack of consistently positive effects on student learning. This article extends the literature by closely examining implementation models and dosage levels for a supplemental software, two gaps in existing research. It also investigates adherence to the core components of the software, and extent to which the supplement enabled personalized instruction. The study was conducted in 40 algebra I classes in an urban school district. Sixty-two percent of classes implemented models that integrated instructional modalities. There was mixed adherence to core components of the software in classes that used it. In the vast majority of classes (94%), software did not enable personalized instruction. Software and the existing curricula were largely independent and did not inform each other. Only one class implemented an integrated instructional model, adhered to the core design components of the software, and demonstrated high levels of personalized instruction. Findings identify implementation barriers and offer suggestions for future implementations and studies of technology-enabled personalization.

[1]  David Boddy,et al.  Prescriptions for managing change: a survey of their effects in projects to implement collaborative working between organisations , 2000 .

[2]  Richard E. Ferdig,et al.  Handbook of Research on K-12 Online and Blended Learning , 2014 .

[3]  Joseph D. Pane,et al.  Informing Progress: Insights on Personalized Learning Implementation and Effects , 2017 .

[4]  Va Alexandria,et al.  Student achievement through staff development (2nd ed. , 1988 .

[5]  Mathea Falco,et al.  A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. , 2003, Health education research.

[6]  Munirpallam A. Venkataramanan,et al.  Enterprise resource planning: Managing the implementation process , 2003, Eur. J. Oper. Res..

[7]  Scotty D. Craig,et al.  Intelligent tutoring systems work as a math gap reducer in 6th grade after-school program , 2016 .

[8]  Steven Ritter,et al.  An experimental study of the effects of Cognitive Tutor® Algebra I on student knowledge and attitude , 2002 .

[9]  T. Hatch,et al.  Crafting Coherence: How Schools Strategically Manage Multiple, External Demands , 2004 .

[10]  Donald P. Ely,et al.  New Perspectives on the Implementation of Educational Technology Innovations. , 1999 .

[11]  Sam Stringfield,et al.  Working Together for Reliable School Reform , 2000 .

[12]  Andrew Booth,et al.  Implementation Science BioMed Central Debate A conceptual framework for implementation fidelity , 2007 .

[13]  E. Rogers.  Diffusion of Innovations , 1962 .

[14]  Dean L. Fixsen,et al.  Improving Programs and Outcomes , 2015 .

[15]  Arthur C. Graesser,et al.  The Effects of a Traditional and Technology-based After-school Setting on 6th Grade Student’s Mathematics Skills , 2012 .

[16]  Patricia Rogers.  Barriers to Adopting Emerging Technologies in Education , 2000 .

[17]  Arthur C. Graesser,et al.  The impact of a technology-based mathematics after-school program using ALEKS on student's knowledge and behaviors , 2013, Comput. Educ..

[18]  Harold H. Wenglinsky.  Does It Compute? The Relationship between Educational Technology and Student Achievement in Mathematics. , 1998 .

[19]  John Watson,et al.  Keeping Pace with K-12 Online & Blended Learning: An Annual Review of Policy and Practice. 10 Year Anniversary Issue. , 2013 .

[20]  Scotty D. Craig,et al.  The Impact of a Mathematical Intelligent Tutoring System on Students' Performance on Standardized High-Stake Tests , 2013, Knowledge Spaces, Applications in Education.

[21]  G. Roehrig,et al.  The Role of Teachers' Beliefs and Knowledge in the Adoption of a Reform-Based Curriculum , 2005 .

[22]  Barry E. Porter,et al.  Factors Contributing to the Successful Implementation of Technology Innovations , 2004, J. Educ. Technol. Soc..

[23]  Carlos Monroy,et al.  Toward a New Approach to the Evaluation of a Digital Curriculum Using Learning Analytics , 2015 .

[24]  'Seeing the Forest for the Trees': Towards a Framework for Effective Change Research in Education , 2001 .

[25]  B. Joyce,et al.  Student Achievement Through Staff Development , 1988 .

[26]  Xiangen Hu,et al.  A Potential Technological Solution for Reducing the Achievement Gap Between White And Black Students , 2013, Knowledge Spaces, Applications in Education.

[27]  J. Schacter.  The Impact of Education Technology on Student Achievement: What the Most Current Research Has To Say. , 1999 .

[28]  John F. Pane,et al.  Examining the implementation of technology-based blended algebra I curriculum at scale , 2016, Educational Technology Research and Development.

[29]  Daniel K. Aladjem.  Examining Comprehensive School Reform , 2006 .

[30]  J. Lowery,et al.  Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science , 2009, Implementation science : IS.

[31]  Daniel F. McCaffrey,et al.  An Experiment to Evaluate the Efficacy of Cognitive Tutor Geometry , 2010 .

[32]  Jennifer M. Brill,et al.  Investigating Practices in Teacher Education that Promote and Inhibit Technology Integration Transfer in Early Career Teachers , 2016 .

[33]  Samuel Ebersole,et al.  Adoption of Computer-Based Instructional Methodologies: A Case Study , 2002 .

[34]  J. Schofield.  A Model of Learned Implementation , 2004 .

[35]  Steven Ritter,et al.  What Evidence Matters? A randomized field trial of Cognitive Tutor Algebra I , 2007, ICCE.

[36]  L. Vygotsky.  Mind in Society: The Development of Higher Psychological Processes: Harvard University Press , 1978 .

[37]  Ryan J. Baxter,et al.  Does the Use of Intelligent Learning and Assessment Software Enhance the Acquisition of Financial Accounting Knowledge , 2011 .

[38]  Anthony S. Bryk,et al.  Coherence : What It Is and Why It Should Guide School Improvement Policy , 2002 .

[39]  Ruth Ellen Hanna,et al.  Predicting student preparedness in calculus , 2006 .

[40]  Daniel F. McCaffrey,et al.  Effectiveness of Cognitive Tutor Algebra I at Scale , 2014 .

[41]  Arthur C. Graesser,et al.  The Impact of Epistemological Beliefs on Student Interactions with an Intelligent Tutoring System , 2014, Intelligent Tutoring Systems.

[42]  R. Slavin.  Mastery Learning Reconsidered , 1987 .

[43]  Barbara Means,et al.  Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies , 2009 .

[44]  Donald P. Ely,et al.  Conditions That Facilitate the Implementation of Educational Technology Innovations , 1990 .

[45]  Anthony S. Bryk,et al.  Trust in Schools: A Core Resource for School Reform. , 2003 .

[46]  Karen A. Blase,et al.  Statewide Implementation of Evidence-Based Programs , 2013 .

[47]  P. Kontos,et al.  Implementation Science BioMed Central Debate , 2009 .

[48]  Andrew V. Dane,et al.  Program integrity in primary and early secondary prevention: are implementation effects out of control? , 1998, Clinical psychology review.

[49]  Thomas A. Brush,et al.  Integrating technology into K-12 teaching and learning: current knowledge gaps and recommendations for future research , 2007 .