Modeling learner’s dynamic knowledge construction procedure and cognitive item difficulty for knowledge tracing

Knowledge tracing (KT) is essential for adaptive learning to obtain learners’ current states of knowledge for the purpose of providing adaptive service. Generally, the knowledge construction procedure is constantly evolving because students dynamically learn and forget over time. Unfortunately, to the best of our knowledge most existing approaches consider only a fragment of the information that relates to learning or forgetting, and the problem of making use of rich information during learners’ learning interactions to achieve more precise prediction of learner performance in KT remains under-explored. Moreover, existing work either neglects the problem difficulty or assumes that it is constant, and this is unrealistic in the actual learning process as problem difficulty affects performance undoubtedly and also varies overtime in terms of the cognitive challenge it presents to individual learners. To this end, we herein propose a novel model, KTM-DLF (Knowledge Tracing Machine by modeling cognitive item Difficulty and Learning and Forgetting), to trace the evolution of each learner’s knowledge acquisition during exercise activities by modeling his or her dynamic knowledge construction procedure and cognitive item difficulty. Specifically, we first specify the concept of cognitive item difficulty and propose a method to model the cognitive item difficulty adaptively based on learners’ learning histories. Then, based on two classical theories (the learning curve theory and the Ebbinghaus forgetting curve theory), we propose a method for modeling learners’ learning and forgetting over time. Finally, the KTM-DLF model is proposed to incorporate learners’ abilities, the cognitive item difficulty, and the two dynamic procedures (learning and forgetting) together. We then use the factorization machine framework to embed features in high dimensions and model pairwise interactions to increase the model’s accuracy. Extensive experiments have been conducted on three public real-world datasets, and the results confirm that our proposed model outperforms the other state-of-the-art educational data mining models.

[1]  Steffen Rendle,et al.  Factorization Machines , 2010, 2010 IEEE International Conference on Data Mining.

[2]  H. Simon,et al.  What makes some problems really hard: Explorations in the problem space of difficulty , 1990, Cognitive Psychology.

[3]  Lars Schmidt-Thieme,et al.  Multi-relational Factorization Models for Student Modeling in Intelligent Tutoring Systems , 2015, 2015 Seventh International Conference on Knowledge and Systems Engineering (KSE).

[4]  Andreas Holzinger,et al.  Insights into Learning Competence Through Probabilistic Graphical Models , 2019, CD-MAKE.

[5]  John R. Anderson,et al.  Cognitive Tutors: Lessons Learned , 1995 .

[6]  John R. Anderson,et al.  Knowledge tracing: Modeling the acquisition of procedural knowledge , 2005, User Modeling and User-Adapted Interaction.

[7]  Ryan Shaun Joazeiro de Baker,et al.  New Potentials for Data-Driven Intelligent Tutoring System Development and Optimization , 2013, AI Mag..

[8]  Pengfei Wang,et al.  Spotting Trip Purposes from Taxi Trajectories , 2017, ACM Trans. Intell. Syst. Technol..

[9]  Shiwei Ye,et al.  AI-Tutor: Generating Tailored Remedial Questions and Answers Based on Cognitive Diagnostic Assessment , 2019, 2019 6th International Conference on Behavioral, Economic and Socio-Cultural Computing (BESC).

[10]  Qing Wang,et al.  Knowledge Tracing with Sequential Key-Value Memory Networks , 2019, SIGIR.

[11]  Kim B. Clark,et al.  Behind the learning curve: a sketch of the learning process , 1991 .

[12]  D. Lakens,et al.  Why Psychologists Should by Default Use Welch's t-test Instead of Student's t-test with Unequal Group Sizes , 2017 .

[13]  Yi Yu,et al.  Deep Knowledge Tracing and Dynamic Student Classification for Knowledge Tracing , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[14]  Yi Sun,et al.  Alternating Recursive Method for Q-matrix Learning , 2014, EDM.

[15]  Noah A. Smith,et al.  Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2016, ACL 2016.

[16]  Xiang Li,et al.  Structured Knowledge Tracing Models for Student Assessment on Coursera , 2016, L@S.

[17]  Xiaolu Xiong,et al.  Submission to the NIPS 2016 Workshop on Machine Learning for Education Estimating student proficiency : Deep learning is not the panacea , 2016 .

[18]  Dit-Yan Yeung,et al.  Dynamic Key-Value Memory Networks for Knowledge Tracing , 2016, WWW.

[19]  Kenneth R. Koedinger,et al.  Performance Factors Analysis - A New Alternative to Knowledge Tracing , 2009, AIED.

[20]  Neil T. Heffernan,et al.  Addressing the assessment challenge with an online system that tutors as it assesses , 2009, User Modeling and User-Adapted Interaction.

[21]  Enhong Chen,et al.  Fuzzy Cognitive Diagnosis for Modelling Examinee Performance , 2018, ACM Trans. Intell. Syst. Technol..

[22]  J. D. L. Torre,et al.  DINA Model and Parameter Estimation: A Didactic , 2009 .

[23]  Shiwei Ye,et al.  Field-Aware Knowledge Tracing Machine by Modelling Students' Dynamic Learning Procedure and Item Difficulty , 2019, 2019 International Conference on Data Mining Workshops (ICDMW).

[24]  Burr Settles,et al.  A Trainable Spaced Repetition Model for Language Learning , 2016, ACL.

[25]  Chun-Kit Yeung,et al.  Deep-IRT: Make Deep Learning Based Knowledge Tracing Explainable Using Item Response Theory , 2019, EDM.

[26]  Michel C. Desmarais,et al.  A review of recent advances in learner and skill modeling in intelligent learning environments , 2012, User Modeling and User-Adapted Interaction.

[27]  James O. Berger,et al.  Bayesian analysis of dynamic item response models in educational testing , 2013, 1304.4441.

[28]  Georg Rasch,et al.  Probabilistic Models for Some Intelligence and Attainment Tests , 1981, The SAGE Encyclopedia of Research Design.

[29]  Walt Detmar Meurers,et al.  Word problems: a review of linguistic and numerical factors contributing to their difficulty , 2015, Front. Psychol..

[30]  Hisashi Kashima,et al.  Knowledge Tracing Machines: Factorization Machines for Knowledge Tracing , 2018, AAAI.

[31]  Klaus D. Kubinger,et al.  Item Difficulty of Multiple Choice Tests Dependant on Different Item Response Formats - an Experiment in Fundamental Research on Psychological Assessment , 2007 .

[32]  Le Wu,et al.  Tracking Knowledge Proficiency of Students with Educational Priors , 2017, CIKM.

[33]  Zachary A. Pardos,et al.  KT-IDEM: introducing item difficulty to the knowledge tracing model , 2011, UMAP'11.

[34]  Fabrice Popineau,et al.  DAS3H: Modeling Student Learning and Forgetting for Optimally Scheduling Distributed Practice of Skills , 2019, EDM.

[35]  Chaitanya Ekanadham,et al.  Back to the basics: Bayesian extensions of IRT outperform neural networks for proficiency estimation , 2016, EDM.

[36]  Tomoko Ohkuma,et al.  Augmenting Knowledge Tracing by Considering Forgetting Behavior , 2019, WWW.

[37]  Hui Xiong,et al.  EKT: Exercise-Aware Knowledge Tracing for Student Performance Prediction , 2019, IEEE Transactions on Knowledge and Data Engineering.

[38]  Sungroh Yoon,et al.  Memory-Augmented Neural Networks for Knowledge Tracing from the Perspective of Learning and Forgetting , 2018, 1805.10768.

[39]  Steffen Rendle,et al.  Factorization Machines with libFM , 2012, TIST.

[40]  Michel C. Desmarais,et al.  Improving Knowledge Tracing Model by Integrating Problem Difficulty , 2018, 2018 IEEE International Conference on Data Mining Workshops (ICDMW).

[41]  Panagiotis Adamopoulos,et al.  What makes a great MOOC? An interdisciplinary analysis of student retention in online courses , 2013, ICIS.

[42]  Leonidas J. Guibas,et al.  Deep Knowledge Tracing , 2015, NIPS.

[43]  Zachary A. Pardos,et al.  Adapting Bayesian Knowledge Tracing to a Massive Open Online Course in edX , 2013, EDM.

[44]  Vincent Aleven,et al.  More Accurate Student Modeling through Contextual Estimation of Slip and Guess Probabilities in Bayesian Knowledge Tracing , 2008, Intelligent Tutoring Systems.

[45]  Yuchun Guo,et al.  Concept-Aware Deep Knowledge Tracing and Exercise Recommendation in an Online Learning System , 2019, EDM.

[46]  Kenneth R. Koedinger,et al.  Learning Factors Analysis - A General Method for Cognitive Model Evaluation and Improvement , 2006, Intelligent Tutoring Systems.

[47]  Fitria Amastini,et al.  Intelligent Tutoring System , 2014 .

[48]  Zhiwei Wang,et al.  Deep Knowledge Tracing with Side Information , 2019, AIED.

[49]  Junlin Zhang,et al.  FAT-DeepFFM: Field Attentive Deep Field-aware Factorization Machine , 2019, ICDM.

[50]  Robert V. Lindsey,et al.  Improving Students’ Long-Term Knowledge Retention Through Personalized Review , 2014, Psychological science.

[51]  Mark D. Reckase,et al.  The Discriminating Power of Items That Measure More Than One Dimension , 1991 .

[52]  Hermann Ebbinghaus (1885) Memory: A Contribution to Experimental Psychology , 2013, Annals of Neurosciences.

[53]  H. Simon,et al.  Why are some problems hard? Evidence from Tower of Hanoi , 1985, Cognitive Psychology.