A Survey of Knowledge Tracing

High-quality education is one of the keys to achieving a more sustainable world. The recent COVID-19 epidemic has triggered the outbreak of online education, which has enabled both students and teachers to learn and teach at home. Meanwhile, it is now possible to record and research a large amount of learning data using online learning platforms in order to offer better intelligent educational services. Knowledge Tracing (KT), which aims to monitor students’ evolving knowledge state, is a fundamental and crucial task to support these intelligent services. Therefore, an increasing amount of research attention has been paid to this emerging area and considerable progress has been made. In this survey, we propose a new taxonomy of existing basic KT models from a technical perspective and provide a comprehensive overview of these models in a systematic manner. In addition, many variants of KT models have been proposed to capture more complete learning process. We then review these variants involved in three phases of the learning process: before, during, and after the student learning, respectively. Moreover, we present several typical applications of KT in different educational scenarios. Finally, we provide some potential directions for future research in this fast-growing field.

[1]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[2]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[3]  Guoliang Li,et al.  Representation Learning From Limited Educational Data With Crowdsourced Labels , 2020, IEEE Transactions on Knowledge and Data Engineering.

[4]  Neil T. Heffernan,et al.  The ASSISTment Builder: Supporting the Life Cycle of Tutoring System Content Creation , 2009, IEEE Transactions on Learning Technologies.

[5]  James M. Royer Theories of the transfer of learning , 1979 .

[6]  Kenneth R. Koedinger,et al.  Performance Factors Analysis - A New Alternative to Knowledge Tracing , 2009, AIED.

[7]  Yanbo Xu,et al.  EEG Helps Knowledge Tracing ! , 2014 .

[8]  Sergio Gomez Colmenarejo,et al.  Hybrid computing using a neural network with dynamic external memory , 2016, Nature.

[9]  Marguerite Wotto The Future High Education Distance Learning in Canada, the United States, and France: Insights From Before COVID-19 Secondary Data Analysis , 2020, Journal of Educational Technology Systems.

[10]  Steffen Rendle,et al.  Factorization Machines , 2010, 2010 IEEE International Conference on Data Mining.

[11]  Enhong Chen,et al.  Neural Cognitive Diagnosis for Intelligent Education Systems , 2019, AAAI.

[12]  Hisashi Kashima,et al.  Knowledge Tracing Machines: Factorization Machines for Knowledge Tracing , 2018, AAAI.

[13]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[14]  Toru Ishida,et al.  Open Smart Classroom: Extensible and Scalable Learning System in Smart Space Using Web Service Technology , 2009, IEEE Transactions on Knowledge and Data Engineering.

[15]  Zachary A. Pardos,et al.  Adapting Bayesian Knowledge Tracing to a Massive Open Online Course in edX , 2013, EDM.

[16]  Ryan Shaun Joazeiro de Baker,et al.  Incorporating Scaffolding and Tutor Context into Bayesian Knowledge Tracing to Predict Inquiry Skill Acquisition , 2013, EDM.

[17]  F. Toates Development and Education , 2009 .

[18]  Mykola Pechenizkiy,et al.  Feedback adaptation in web-based learning systems , 2007 .

[19]  Geoffrey R. Loftus,et al.  Evaluating forgetting curves. , 1985 .

[20]  Gerhard Fischer,et al.  User Modeling in Human–Computer Interaction , 2001, User Modeling and User-Adapted Interaction.

[21]  George Karypis,et al.  A Self Attentive model for Knowledge Tracing , 2019, EDM.

[22]  Zachary A. Pardos,et al.  Clustered Knowledge Tracing , 2012, ITS.

[23]  E. Emanuel Online education: MOOCs taken by educated few , 2013, Nature.

[24]  Haiqin Yang,et al.  Heterogeneous Features Integration in Deep Knowledge Tracing , 2017, ICONIP.

[25]  Nichola Gretton A learner centred approach to online education , 2015 .

[26]  Vincent Aleven,et al.  Educational Game and Intelligent Tutoring System , 2017, ACM Trans. Comput. Hum. Interact..

[27]  Meng Wang,et al.  Using Feedback to Improve Learning: Differentiating between Correct and Erroneous Examples , 2015, 2015 International Symposium on Educational Technology (ISET).

[28]  Michel C. Desmarais,et al.  A review of recent advances in learner and skill modeling in intelligent learning environments , 2012, User Modeling and User-Adapted Interaction.

[29]  Radek Pelánek,et al.  Application of Time Decay Functions and the Elo System in Student Modeling , 2014, EDM.

[30]  K. VanLehn The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems, and Other Tutoring Systems , 2011 .

[31]  Zachary A. Pardos,et al.  Modeling Individualization in a Bayesian Networks Implementation of Knowledge Tracing , 2010, UMAP.

[32]  Dit-Yan Yeung,et al.  Addressing two problems in deep knowledge tracing via prediction-consistent regularization , 2018, L@S.

[33]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[34]  Enhong Chen,et al.  Finding Similar Exercises in Online Education Systems , 2018, KDD.

[35]  Tuan D. Nguyen,et al.  The Effectiveness of Online Learning : Beyond No Significant Difference and Future Horizons , 2015 .

[36]  Dit-Yan Yeung,et al.  Dynamic Key-Value Memory Networks for Knowledge Tracing , 2016, WWW.

[37]  Vicki Trowler Student engagement literature review , 2010 .

[38]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[39]  Chen Lin,et al.  Intervention-BKT: Incorporating Instructional Interventions into Bayesian Knowledge Tracing , 2016, ITS.

[40]  George D. Kuh,et al.  Student Engagement and Student Learning: Testing the Linkages* , 2006 .

[41]  Byungsoo Kim,et al.  Towards an Appropriate Query, Key, and Value Computation for Knowledge Tracing , 2020, L@S.

[42]  Lars Schmidt-Thieme,et al.  Using factorization machines for student modeling , 2012, UMAP Workshops.

[43]  Enhong Chen,et al.  QuesNet: A Unified Representation for Heterogeneous Test Questions , 2019, KDD.

[44]  Michelle N. Lumicao,et al.  EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. , 2007, Aviation, space, and environmental medicine.

[45]  Enhong Chen,et al.  Exploiting Cognitive Structure for Adaptive Learning , 2019, KDD.

[46]  Jian Shen,et al.  Improving Knowledge Tracing via Pre-training Question Embeddings , 2020, IJCAI.

[47]  Richard Lesh,et al.  Children’s Mathematical Reasoning in Online Games: Can Data Mining Reveal Strategic Thinking? , 2011 .

[48]  Ivon Arroyo,et al.  Modeling the Interplay Between Knowledge and Affective Engagement in Students , 2014, Int. J. People Oriented Program..

[49]  Radek Pelánek,et al.  Modeling Students' Memory for Application in Adaptive Educational Systems , 2015, EDM.

[50]  Joseph E. Beck,et al.  Using Knowledge Tracing in a Noisy Environment to Measure Student Reading Proficiencies , 2006, Int. J. Artif. Intell. Educ..

[51]  Kenneth R. Koedinger,et al.  Learning Factors Analysis - A General Method for Cognitive Model Evaluation and Improvement , 2006, Intelligent Tutoring Systems.

[52]  Enhong Chen,et al.  Question Difficulty Prediction for READING Problems in Standard Tests , 2017, AAAI.

[53]  Stefan Kopp,et al.  Adaptive Robot Language Tutoring Based on Bayesian Knowledge Tracing and Predictive Decision-Making , 2017, 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI.

[54]  Philip S. Yu,et al.  Anonymizing Classification Data for Privacy Preservation , 2007, IEEE Transactions on Knowledge and Data Engineering.

[55]  Qing Wang,et al.  Knowledge Tracing with Sequential Key-Value Memory Networks , 2019, SIGIR.

[56]  Peter Brusilovsky,et al.  General Features in Knowledge Tracing to Model Multiple Subskills, Temporal Item Response Theory, and Expert Knowledge , 2014, EDM.

[57]  Byungsoo Kim,et al.  EdNet: A Large-Scale Hierarchical Dataset in Education , 2020, AIED.

[58]  Cord Hockemeyer,et al.  Automata for the Assessment of Knowledge , 2001, IEEE Trans. Knowl. Data Eng..

[59]  Jaideep Srivastava,et al.  RKT: Relation-Aware Self-Attention for Knowledge Tracing , 2020, CIKM.

[60]  Neil T. Heffernan,et al.  Incorporating Rich Features into Deep Knowledge Tracing , 2017, L@S.

[61]  Jia Zhu,et al.  Learning from Interpretable Analysis: Attention-Based Knowledge Tracing , 2020, AIED.

[62]  Enhong Chen,et al.  Structure-based Knowledge Tracing: An Influence Propagation View , 2020, 2020 IEEE International Conference on Data Mining (ICDM).

[63]  Enhong Chen,et al.  Federated Deep Knowledge Tracing , 2021, WSDM.

[64]  Enhong Chen,et al.  Constructing Educational Concept Maps with Multiple Relationships from Multi-Source Data , 2019, 2019 IEEE International Conference on Data Mining (ICDM).

[65]  John R. Anderson,et al.  Knowledge tracing: Modeling the acquisition of procedural knowledge , 2005, User Modeling and User-Adapted Interaction.

[66]  Emma Brunskill,et al.  The Impact on Individualizing Student Models on Necessary Practice Opportunities , 2012, EDM.

[67]  Yi Yu,et al.  Deep Knowledge Tracing and Dynamic Student Classification for Knowledge Tracing , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[68]  Enhong Chen,et al.  Fuzzy Cognitive Diagnosis for Modelling Examinee Performance , 2018, ACM Trans. Intell. Syst. Technol..

[69]  Anders D. Olofsson,et al.  A study of the use of digital technology and its conditions with a view to understanding what ‘adequate digital competence’ may mean in a national policy initiative , 2020, Educational Studies.

[70]  Prema Nedungadi,et al.  Incorporating forgetting in the Personalized, Clustered, Bayesian Knowledge Tracing (PC-BKT) model , 2015, 2015 International Conference on Cognitive Computing and Information Processing(CCIP).

[71]  Alexander G. Schwing,et al.  Dynamic Bayesian Networks for Student Modeling , 2017, IEEE Transactions on Learning Technologies.

[72]  Changqin Huang,et al.  Fine-grained learning performance prediction via adaptive sparse self-attention networks , 2021, Inf. Sci..

[73]  Neil T. Heffernan,et al.  Leveraging First Response Time into the Knowledge Tracing Model , 2012, EDM.

[74]  David Lang,et al.  Deep Knowledge Tracing and Engagement with MOOCs , 2019, LAK.

[75]  Aritra Ghosh,et al.  Context-Aware Attentive Knowledge Tracing , 2020, KDD.

[76]  Hui Xiong,et al.  Personalized Travel Package Recommendation , 2011, 2011 IEEE 11th International Conference on Data Mining.

[77]  Dongmin Shin,et al.  SAINT+: Integrating Temporal Features for EdNet Correctness Prediction , 2020, LAK.

[78]  John DeNero,et al.  Painless Unsupervised Learning with Features , 2010, NAACL.

[79]  Enhong Chen,et al.  Exploring Multi-Objective Exercise Recommendations in Online Education Systems , 2019, CIKM.

[80]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[81]  Jichen Zhu,et al.  Tracing Player Knowledge in a Parallel Programming Educational Game , 2018, AIIDE.

[82]  Zachary A. Pardos,et al.  Clustering Students to Generate an Ensemble to Improve Standard Test Score Predictions , 2011, AIED.

[83]  Ivon Arroyo,et al.  Tracing Knowledge and Engagement in Parallel in an Intelligent Tutoring System , 2014, EDM.

[84]  Kenneth R. Koedinger,et al.  Individualized Bayesian Knowledge Tracing Models , 2013, AIED.

[85]  Qian Wang,et al.  Differentially Private Distributed Online Learning , 2015, IEEE Transactions on Knowledge and Data Engineering.

[86]  Joseph E. Beck,et al.  Going Deeper with Deep Knowledge Tracing , 2016, EDM.

[87]  Bin Jiang,et al.  Clustering Uncertain Data Based on Probability Distribution Similarity , 2013, IEEE Transactions on Knowledge and Data Engineering.

[88]  E. Snow,et al.  The New Five-Year Federal Strategic Plan in STEM Education: What’s in it for Science? , 2018 .

[89]  Le Wu,et al.  Tracking Knowledge Proficiency of Students with Educational Priors , 2017, CIKM.

[90]  Leonidas J. Guibas,et al.  Deep Knowledge Tracing , 2015, NIPS.

[91]  Zachary A. Pardos,et al.  Does Time Matter? Modeling the Effect of Time with Bayesian Knowledge Tracing , 2011, EDM.

[92]  Tomoko Ohkuma,et al.  Augmenting Knowledge Tracing by Considering Forgetting Behavior , 2019, WWW.

[93]  Lars Schmidt-Thieme,et al.  Factorization Techniques for Predicting Student Performance , 2012 .

[94]  Zi Huang,et al.  Discrete Nonnegative Spectral Clustering , 2017, IEEE Transactions on Knowledge and Data Engineering.

[95]  F. Lord Applications of Item Response Theory To Practical Testing Problems , 1980 .

[96]  Qi Liu,et al.  Learning or Forgetting? A Dynamic Approach for Tracking the Knowledge Proficiency of Students , 2020, ACM Trans. Inf. Syst..

[97]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[98]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[99]  Shaul Markovitch,et al.  The Role of Forgetting in Learning , 1988, ML.

[100]  Zachary MacHardy,et al.  Applications of Bayesian Knowledge Tracing to the Curation of Educational Videos , 2015 .

[101]  Hui Xiong,et al.  EKT: Exercise-Aware Knowledge Tracing for Student Performance Prediction , 2019, IEEE Transactions on Knowledge and Data Engineering.

[102]  Enhong Chen,et al.  Convolutional Knowledge Tracing: Modeling Individualization in Student Learning Process , 2020, SIGIR.

[103]  Daniele Loiacono,et al.  Player Modeling , 2013, Artificial and Computational Intelligence in Games.

[104]  Albert T. Corbett,et al.  Does Help Help? Introducing the Bayesian Evaluation and Assessment Methodology , 2008, Intelligent Tutoring Systems.

[105]  Enhong Chen,et al.  Exercise-Enhanced Sequential Modeling for Student Performance Prediction , 2018, AAAI.

[106]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[107]  Yiqun Liu,et al.  Temporal Cross-Effects in Knowledge Tracing , 2021, WSDM.

[108]  Radek Pelánek,et al.  Adaptive Practice of Facts in Domains with Varied Prior Knowledge , 2014, EDM.

[109]  Radek Pelánek,et al.  Bayesian knowledge tracing, logistic models, and beyond: an overview of learner modeling techniques , 2017, User Modeling and User-Adapted Interaction.

[110]  Yiannis Demiris,et al.  Online Knowledge Level Tracking with Data-Driven Student Models and Collaborative Filtering , 2020, IEEE Transactions on Knowledge and Data Engineering.