Effectiveness of Cognitive Tutor Algebra I at Scale

This article examines the effectiveness of a technology-based algebra curriculum in a wide variety of middle schools and high schools in seven states. Participating schools were matched into similar pairs and randomly assigned to either continue with the current algebra curriculum for 2 years or to adopt Cognitive Tutor Algebra I (CTAI), which uses a personalized, mastery-learning, blended-learning approach. Schools assigned to implement CTAI did so under conditions similar to schools that independently adopt it. Analysis of posttest outcomes on an algebra proficiency exam finds no effects in the first year of implementation, but finds evidence in support of positive effects in the second year. The estimated effect is statistically significant for high schools but not for middle schools; in both cases, the magnitude is sufficient to improve the median student’s performance by approximately eight percentile points.

[1]  Austin Lasseter,et al.  Understanding the Implications of Online Learning for Educational Productivity , 2012 .

[2]  J. Schacter.  The Impact of Education Technology on Student Achievement: What the Most Current Research Has To Say. , 1999 .

[3]  Albert T. Corbett,et al.  Cognitive Tutor: Applied research in mathematics education , 2007, Psychonomic bulletin & review.

[4]  Raymond J. Carroll,et al.  Measurement error in nonlinear models: a modern perspective , 2006 .

[5]  Daniel F. McCaffrey,et al.  An Experiment to Evaluate the Efficacy of Cognitive Tutor Geometry , 2010 .

[6]  John F. Pane,et al.  Analysis of Costs in an Algebra I Curriculum Effectiveness Study. Technical Report. , 2012 .

[7]  W. Greene,et al.  计量经济分析 = Econometric analysis , 2009 .

[8]  Roberto Agodini,et al.  Effectiveness of Reading and Mathematics Software Products: Findings From Two Student Cohorts. NCEE 2009-4041. , 2009 .

[9]  Karen B. Givvin,et al.  Mathematics Teaching in the United States Today (and Tomorrow): Results From the TIMSS 1999 Video Study , 2005 .

[10]  W. Ray,et al.  Performance of disease risk scores, propensity scores, and traditional multivariable outcome regression in the presence of multiple confounders. , 2011, American journal of epidemiology.

[11]  B. Hansen.  The prognostic analogue of the propensity score , 2008 .

[12]  Mark W. Lipsey,et al.  Translating the Statistical Representation of the Effects of Education Interventions Into More Readily Interpretable Forms , 2012 .

[13]  M. Kenward,et al.  An Introduction to the Bootstrap , 2007 .

[14]  Steven Ritter,et al.  What Evidence Matters? A randomized field trial of Cognitive Tutor Algebra I , 2007, ICCE.

[15]  A. Su,et al.  The National Council of Teachers of Mathematics , 1932, The Mathematical Gazette.

[16]  Lena Osterhagen,et al.  Multiple Imputation For Nonresponse In Surveys , 2016 .

[17]  I. Mullis,et al.  TIMSS 2011 International Results in Mathematics. , 2012 .

[18]  Daniel F. McCaffrey,et al.  Correcting for Test Score Measurement Error in ANCOVA Models for Estimating Treatment Effects , 2014 .

[19]  최영한,et al.  미국 NCTM의 Principles and Standards for School Mathematics에 나타난 수학과 교수,학습의 이론 , 2002 .

[20]  John R. Anderson,et al.  Cognitive Tutors: Lessons Learned , 1995 .

[21]  What Works Clearinghouse: Procedures and Standards Handbook (Version 2.1) , 2011 .

[22]  E. Fennema,et al.  Fennema-Sherman Mathematics Attitudes Scales: Instruments Designed to Measure Attitudes Toward the Learning of Mathematics by Females and Males. , 1976 .

[23]  Robert E. Ployhart,et al.  Hierarchical Linear Models , 2014 .

[24]  Steven Ritter,et al.  An experimental study of the effects of Cognitive Tutor® Algebra I on student knowledge and attitude , 2002 .

[25]  J. Gill.  Hierarchical Linear Models , 2005 .

[26]  O S Miettinen,et al.  Stratification by a multivariate confounder score. , 1976, American journal of epidemiology.

[27]  Harold H. Wenglinsky.  Does It Compute? The Relationship between Educational Technology and Student Achievement in Mathematics. , 1998 .

[28]  Boya Ma,et al.  Comparative Effectiveness of Carnegie Learning's "Cognitive Tutor Bridge to Algebra" Curriculum: A Report of a Randomized Experiment in the Maui School District. Research Report. , 2007 .

[29]  Karen B Givvin,et al.  Mathematics Teaching in the United States Today (and Tomorrow): Results from the TIMSS 1999 Video Study , 2006 .

[30]  D. Hall.  Measurement Error in Nonlinear Models: A Modern Perspective , 2008 .

[31]  D. Rubin,et al.  Small-sample degrees of freedom with multiple imputation , 1999 .

[32]  Roberto Agodini,et al.  Effectiveness of Reading and Mathematics Software Products: Findings from Two Student Cohorts. Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education , 2009 .

[33]  Barbara Means,et al.  Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies , 2009 .

[34]  Arthur C. Graesser,et al.  Organizing Instruction and Study to Improve Student Learning. IES Practice Guide. NCER 2007-2004. , 2007 .

[35]  Lynn S. Fuchs,et al.  Using Curriculum‐Based Measurement to Improve Student Achievement: Review of Research , 2005 .