Dynamic Cognitive Tracing: Towards Unified Discovery of Student and Cognitive Models

This work describes a unified approach to two problems previously addressed separately in Intelligent Tutoring Systems: (i) Cognitive Modeling, which factorizes problem solving steps into the latent set of skills required to perform them [7]; and (ii) Student Modeling, which infers students’ learning by observing student performance [9]. The practical importance of improving understanding of how students learn is to build better intelligent tutors [8]. The expected advantages of our integrated approach include (i) more accurate prediction of a student’s future performance, and (ii) clustering items into skills automatically, without expensive manual expert knowledge annotation. We introduce a unified model, Dynamic Cognitive Tracing, to explain student learning in terms of skill mastery over time, by learning the Cognitive Model and the Student Model jointly. We formulate our approach as a graphical model, and we validate it using sixty different synthetic datasets. Dynamic Cognitive Tracing significantly outperforms single-skill Knowledge Tracing on predicting future student performance.

[1]  Lars Schmidt-Thieme,et al.  Recommender system for predicting student performance , 2010, RecSysTEL@RecSys.

[2]  Michel C. Desmarais Conditions for Effectively Deriving a Q-Matrix from Data with Non-negative Matrix Factorization. Best Paper Award , 2011, EDM.

[3]  Kenneth R. Koedinger,et al.  Performance Factors Analysis - A New Alternative to Knowledge Tracing , 2009, AIED.

[4]  Michael I. Jordan,et al.  Factorial Hidden Markov Models , 1995, Machine Learning.

[5]  John R. Anderson,et al.  Knowledge tracing: Modeling the acquisition of procedural knowledge , 2005, User Modeling and User-Adapted Interaction.

[6]  Kenneth R. Koedinger,et al.  A Data Repository for the EDM Community: The PSLC DataShop , 2010 .

[7]  Mladen A. Vouk,et al.  Experimental Analysis of the Q-Matrix Method in Knowledge Discovery , 2005, ISMIS.

[8]  Alex Bateman,et al.  An introduction to hidden Markov models. , 2007, Current protocols in bioinformatics.

[9]  Tiffany Barnes,et al.  The Q-matrix Method: Mining Student Response Data for Knowledge , 2005 .

[10]  Joseph E. Beck,et al.  Identifiability: A Fundamental Problem of Student Modeling , 2007, User Modeling.

[11]  Vincent Aleven,et al.  More Accurate Student Modeling through Contextual Estimation of Slip and Guess Probabilities in Bayesian Knowledge Tracing , 2008, Intelligent Tutoring Systems.

[12]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[13]  J. Beck Difficulties in inferring student knowledge from observations ( and why you should care ) , 2007 .

[14]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[15]  Kevin Murphy,et al.  Bayes net toolbox for Matlab , 1999 .

[16]  Jim Reye,et al.  Student Modelling Based on Belief Networks , 2004, Int. J. Artif. Intell. Educ..

[17]  Jack Mostow,et al.  How Who Should Practice: Using Learning Decomposition to Evaluate the Efficacy of Different Types of Practice for Different Types of Students , 2008, Intelligent Tutoring Systems.

[18]  Lawrence K. Saul,et al.  A Generalized Linear Model for Principal Component Analysis of Binary Data , 2003, AISTATS.

[19]  Neil T. Heffernan,et al.  Comparing Knowledge Tracing and Performance Factor Analysis by Using Multiple Model Fitting Procedures , 2010, Intelligent Tutoring Systems.

[20]  Geoffrey J. Gordon,et al.  A Unified View of Matrix Factorization Models , 2008, ECML/PKDD.

[21]  Thomas L. Griffiths,et al.  A fully Bayesian approach to unsupervised part-of-speech tagging , 2007, ACL.

[22]  Robert A. Jacobs,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1993, Neural Computation.

[23]  Yanbo Xu,et al.  Using Logistic Regression to Trace Multiple Sub-skills in a Dynamic Bayes Net , 2011, EDM.

[24]  Kenneth R. Koedinger,et al.  Learning Factors Analysis - A General Method for Cognitive Model Evaluation and Improvement , 2006, Intelligent Tutoring Systems.

[25]  Michael E. Tipping,et al.  Probabilistic Principal Component Analysis , 1999 .

[26]  Albert T. Corbett,et al.  Cognitive Computer Tutors: Solving the Two-Sigma Problem , 2001, User Modeling.