EKT: Exercise-Aware Knowledge Tracing for Student Performance Prediction

For offering proactive services (e.g., personalized exercise recommendation) to the students in computer supported intelligent education, one of the fundamental tasks is predicting student performance (e.g., scores) on future exercises, where it is necessary to track the change of each student's knowledge acquisition during her exercising activities. Unfortunately, to the best of our knowledge, existing approaches can only exploit the exercising records of students, and the problem of extracting rich information existed in the materials (e.g., knowledge concepts, exercise content) of exercises to achieve both more precise prediction of student performance and more interpretable analysis of knowledge acquisition remains underexplored. To this end, in this paper, we present a holistic study of student performance prediction. To directly achieve the primary goal of performance prediction, we first propose a general Exercise-Enhanced Recurrent Neural Network (EERNN) framework by exploring both student's exercising records and the text content of corresponding exercises. In EERNN, we simply summarize each student's state into an integrated vector and trace it with a recurrent neural network, where we design a bidirectional LSTM to learn the encoding of each exercise from its content. For making final predictions, we design two implementations on the basis of EERNN with different prediction strategies, i.e., EERNNM with Markov property and EERNNA with Attention mechanism. Then, to explicitly track student's knowledge acquisition on multiple knowledge concepts, we extend EERNN to an explainable Exercise-aware Knowledge Tracing (EKT) framework by incorporating the knowledge concept information, where the student's integrated state vector is now extended to a knowledge state matrix. In EKT, we further develop a memory network for quantifying how much each exercise can affect the mastery of students on multiple knowledge concepts during the exercising process. Finally, we conduct extensive experiments and evaluate both EERNN and EKT frameworks on a large-scale real-world data. The results in both general and cold-start scenarios clearly demonstrate the effectiveness of two frameworks in student performance prediction as well as the superior interpretability of EKT.

[1]  Wenwu Zhu,et al.  Deep Variational Network Embedding in Wasserstein Space , 2018, KDD.

[2]  Lars Schmidt-Thieme,et al.  Recommender system for predicting student performance , 2010, RecSysTEL@RecSys.

[3]  Steven P. Reise,et al.  Item Response Theory , 2014 .

[4]  Kenneth R. Koedinger,et al.  Individualized Bayesian Knowledge Tracing Models , 2013, AIED.

[5]  Xing Xie,et al.  Transcribing Content from Structural Images with Spotlight Mechanism , 2018, KDD.

[6]  Richard Socher,et al.  Ask Me Anything: Dynamic Memory Networks for Natural Language Processing , 2015, ICML.

[7]  John R. Anderson,et al.  Knowledge tracing: Modeling the acquisition of procedural knowledge , 2005, User Modeling and User-Adapted Interaction.

[8]  Liang Zhang,et al.  CADEN: A Context-Aware Deep Embedding Network for Financial Opinions Mining , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[9]  Penghe Chen,et al.  Prerequisite-Driven Deep Knowledge Tracing , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[10]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[11]  Enhong Chen,et al.  Question Difficulty Prediction for READING Problems in Standard Tests , 2017, AAAI.

[12]  Qi Liu,et al.  Voice of Charity: Prospecting the Donation Recurrence & Donor Retention in Crowdfunding , 2020, IEEE Transactions on Knowledge and Data Engineering.

[13]  Daan Wierstra,et al.  Meta-Learning with Memory-Augmented Neural Networks , 2016, ICML.

[14]  Le Wu,et al.  Tracking Knowledge Proficiency of Students with Educational Priors , 2017, CIKM.

[15]  Jure Leskovec,et al.  Engaging with massive online courses , 2014, WWW.

[16]  Leonidas J. Guibas,et al.  Deep Knowledge Tracing , 2015, NIPS.

[17]  Ryan Shaun Joazeiro de Baker,et al.  Case studies in the use of ROC curve analysis for sensor-based estimates in human computer interaction , 2005, Graphics Interface.

[18]  Kenneth R. Koedinger,et al.  Performance Factors Analysis - A New Alternative to Knowledge Tracing , 2009, AIED.

[19]  Enhong Chen,et al.  Exercise-Enhanced Sequential Modeling for Student Performance Prediction , 2018, AAAI.

[20]  A. A. G. Putra,et al.  PIECING TOGETHER THE STUDENT SUCCESS PUZZLE: RESEARCH, PROPOSITIONS, AND RECOMMENDATIONS , 2013 .

[21]  Hui Xiong,et al.  A Cocktail Approach for Travel Package Recommendation , 2014, IEEE Transactions on Knowledge and Data Engineering.

[22]  Enhong Chen,et al.  Knowledge or Gaming?: Cognitive Modelling Based on Multiple-Attempt Response , 2017, WWW.

[23]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[24]  Zachary A. Pardos,et al.  KT-IDEM: introducing item difficulty to the knowledge tracing model , 2011, UMAP'11.

[25]  Bo Li,et al.  Stable Prediction across Unknown Environments , 2018, KDD.

[26]  Richard Socher,et al.  Dynamic Memory Networks for Visual and Textual Question Answering , 2016, ICML.

[27]  Jian Pei,et al.  A Survey on Network Embedding , 2017, IEEE Transactions on Knowledge and Data Engineering.

[28]  Peter Brusilovsky,et al.  Integrating Knowledge Tracing and Item Response Theory: A Tale of Two Frameworks , 2014, UMAP Workshops.

[29]  Ryan S. Baker,et al.  The State of Educational Data Mining in 2009: A Review and Future Visions. , 2009, EDM 2009.

[30]  J. van Leeuwen,et al.  Neural Networks: Tricks of the Trade , 2002, Lecture Notes in Computer Science.

[31]  Lars Schmidt-Thieme,et al.  Factorization Techniques for Predicting Student Performance , 2012 .

[32]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[33]  Yanbo Xu,et al.  Using Logistic Regression to Trace Multiple Sub-skills in a Dynamic Bayes Net , 2011, EDM.

[34]  Liang Wang,et al.  Multi-Behavioral Sequential Prediction with Recurrent Log-Bilinear Model , 2016, IEEE Transactions on Knowledge and Data Engineering.

[35]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[36]  Dit-Yan Yeung,et al.  Dynamic Key-Value Memory Networks for Knowledge Tracing , 2016, WWW.

[37]  Eduardo Salas,et al.  The Transfer of Training: What Really Matters , 2011 .

[38]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[39]  Yuefan Deng,et al.  Applying Simulated Annealing and Parallel Computing to the Mobile Sequential Recommendation , 2019, IEEE Transactions on Knowledge and Data Engineering.

[40]  Christine E. DeMars,et al.  Item Response Theory , 2010, Assessing Measurement Invariance for Applied Research.

[41]  Huan Liu,et al.  Interactive Unknowns Recommendation in E-Learning Systems , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[42]  Bowen Zhou,et al.  LSTM-based Deep Learning Models for non-factoid answer selection , 2015, ArXiv.

[43]  Louis V. DiBello,et al.  31A Review of Cognitively Diagnostic Assessment and a Summary of Psychometric Models , 2006 .

[44]  J. D. L. Torre,et al.  DINA Model and Parameter Estimation: A Didactic , 2009 .

[45]  Crystal Rion,et al.  Piecing Together the Student Success Puzzle: Research, Propositions and Recommendations (review) , 2007 .

[46]  Robert V. Lindsey,et al.  Incorporating Latent Factors Into Knowledge Tracing To Predict Individual Differences In Learning , 2013 .

[47]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[48]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[49]  Sergio Gomez Colmenarejo,et al.  Hybrid computing using a neural network with dynamic external memory , 2016, Nature.

[50]  Enhong Chen,et al.  Cognitive Modelling for Predicting Examinee Performance , 2015, IJCAI.

[51]  Grgoire Montavon,et al.  Neural Networks: Tricks of the Trade , 2012, Lecture Notes in Computer Science.

[52]  Huan Liu,et al.  Exploring Hierarchical Structures for Recommender Systems , 2018, IEEE Transactions on Knowledge and Data Engineering.

[53]  Enhong Chen,et al.  Fuzzy Cognitive Diagnosis for Modelling Examinee Performance , 2018, ACM Trans. Intell. Syst. Technol..

[54]  Panos Kalnis,et al.  Searching Trajectories by Regions of Interest , 2017, IEEE Transactions on Knowledge and Data Engineering.

[55]  Kenneth R. Koedinger,et al.  Learning Factors Analysis - A General Method for Cognitive Model Evaluation and Improvement , 2006, Intelligent Tutoring Systems.

[56]  Xiaolu Xiong,et al.  Submission to the NIPS 2016 Workshop on Machine Learning for Education Estimating student proficiency : Deep learning is not the panacea , 2016 .

[57]  Alex Bateman,et al.  An introduction to hidden Markov models. , 2007, Current protocols in bioinformatics.

[58]  Xing Xie,et al.  Automatically Learning Topics and Difficulty Levels of Problems in Online Judge Systems , 2018, ACM Trans. Inf. Syst..

[59]  Shaowei Liu,et al.  General Knowledge Embedded Image Representation Learning , 2018, IEEE Transactions on Multimedia.

[60]  Omer Levy,et al.  word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method , 2014, ArXiv.

[61]  Jason Weston,et al.  End-To-End Memory Networks , 2015, NIPS.

[62]  Michael Jahrer,et al.  Collaborative Filtering Applied to Educational Data Mining , 2010 .

[63]  Xiangmin Xu,et al.  Hierarchical Lifelong Learning by Sharing Representations and Integrating Hypothesis , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[64]  Richard G. Baraniuk,et al.  Time-varying learning and content analytics via sparse factor analysis , 2013, KDD.

[65]  Junping Du,et al.  A Sequential Approach to Market State Modeling and Analysis in Online P2P Lending , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.