Integrating latent-factor and knowledge-tracing models to predict individual differences in learning

An eective tutor|human or digital|must determine what a student does and does not know. Inferring a student’s knowledge state is challenging because behavioral observations (e.g., correct vs. incorrect problem solution) provide only weak evidence. Two classes of models have been proposed to address the challenge. Latent-factor models employ a collaborative ltering approach in which data from a population of students solving a population of problems is used to predict the performance of an individual student on a specic problem. Knowledge-tracing models exploit a student’s sequence of problem-solving attempts to determine the point at which a skill is mastered. Although these two approaches are complementary, only preliminary, informal steps have been taken to integrate them. We propose a principled synthesis of the two approaches in a hierarchical Bayesian model that predicts student performance by integrating a theory of the temporal dynamics of learning with a theory of individual dierences among students and problems. We present results from three data sets from the DataShop repository indicating that the integrated architecture outperforms either alone. We nd signicant predictive value in considering the diculty of specic problems (within a skill), a source of information that has rarely been exploited.

[1]  R. Atkinson Optimizing the Learning of a Second-Language Vocabulary. , 1972 .

[2]  P. Boeck,et al.  Explanatory item response models : a generalized linear and nonlinear approach , 2004 .

[3]  John R. Anderson,et al.  Knowledge tracing: Modeling the acquisition of procedural knowledge , 2005, User Modeling and User-Adapted Interaction.

[4]  Kenneth R. Koedinger,et al.  Learning Factors Analysis - A General Method for Cognitive Model Evaluation and Improvement , 2006, Intelligent Tutoring Systems.

[5]  J. Beck Difficulties in inferring student knowledge from observations ( and why you should care ) , 2007 .

[6]  Joseph E. Beck,et al.  Identifiability: A Fundamental Problem of Student Modeling , 2007, User Modeling.

[7]  Vincent Aleven,et al.  More Accurate Student Modeling through Contextual Estimation of Slip and Guess Probabilities in Bayesian Knowledge Tracing , 2008, Intelligent Tutoring Systems.

[8]  Kenneth R. Koedinger,et al.  Comparing Two IRT Models for Conjunctive Skills , 2008, Intelligent Tutoring Systems.

[9]  Kenneth R. Koedinger,et al.  Performance Factors Analysis - A New Alternative to Knowledge Tracing , 2009, AIED.

[10]  Kenneth R. Koedinger,et al.  Generalized learning factors analysis: improving cognitive models with machine learning , 2009 .

[11]  Shou-De Lin,et al.  Feature Engineering and Classifier Ensemble for KDD Cup 2010 , 2010, KDD 2010.

[12]  Zachary A. Pardos,et al.  Modeling Individualization in a Bayesian Networks Implementation of Knowledge Tracing , 2010, UMAP.

[13]  Ryan Shaun Joazeiro de Baker,et al.  Contextual Slip and Prediction of Student Performance after Use of an Intelligent Tutor , 2010, UMAP.

[14]  Kenneth R. Koedinger,et al.  A Data Repository for the EDM Community: The PSLC DataShop , 2010 .

[15]  Zachary A. Pardos,et al.  KT-IDEM: introducing item difficulty to the knowledge tracing model , 2011, UMAP'11.

[16]  Jonathan P. Rowe,et al.  Improving Models of Slipping, Guessing, and Moment-By-Moment Learning with Estimates of Skill Difficulty , 2011, EDM.

[17]  Kurt VanLehn,et al.  Instructional Factors Analysis: A Cognitive Model For Multiple Instructional Interventions , 2011, EDM.

[18]  Emma Brunskill,et al.  The Impact on Individualizing Student Models on Necessary Practice Opportunities , 2012, EDM.

[19]  Zachary A. Pardos,et al.  Clustered Knowledge Tracing , 2012, ITS.

[20]  Kenneth R. Koedinger,et al.  Individualized Bayesian Knowledge Tracing Models , 2013, AIED.

[21]  Yanbo Xu,et al.  Using Item Response Theory to Refine Knowledge Tracing , 2013, EDM.

[22]  Peter Brusilovsky,et al.  General Features in Knowledge Tracing to Model Multiple Subskills, Temporal Item Response Theory, and Expert Knowledge , 2014, EDM.

[23]  Peter Brusilovsky,et al.  Integrating Knowledge Tracing and Item Response Theory: A Tale of Two Frameworks , 2014, UMAP Workshops.

[24]  Robert V. Lindsey,et al.  Improving Students’ Long-Term Knowledge Retention Through Personalized Review , 2014, Psychological science.