CMKT: Concept Map Driven Knowledge Tracing

In this article, we advocate for and propose a novel concept map driven knowledge tracing (CMKT) model, which utilizes educational concept map for learner modeling. This article particularly addresses the issue of learner data sparseness caused by the unwillingness to practice and irregular learning behaviors on the learner side. CMKT considers the concept map as a new information source and explicitly exploits its inherent information to help the estimation of the learner's knowledge state. Specifically, the pairwise educational relations in the concept map are formulated as the ordering pairs and are used as mathematical constraints for model construction. The topology information in the concept map is extracted and used as the model input by employing the network embedding techniques. Integrating both educational relation information and topology information in the concept map, CMKT adopts the recurrent neural network to perform knowledge tracing tasks. Comprehensive evaluations conducted on five public educational datasets of four different subjects (more than 8000 learners and their 300 000 records) demonstrate the promise and effectiveness of CMKT: The average area under ROC curve (AUC) and overall prediction accuracy (ACC) achieve 0.82 and 0.75, respectively, and CMKT outperforms all the baselines by at least 12.2% and 9.2% in terms of AUC and ACC.

[1]  Enhong Chen,et al.  Structure-based Knowledge Tracing: An Influence Propagation View , 2020, 2020 IEEE International Conference on Data Mining (ICDM).

[2]  Daniel K Hartline,et al.  t-Distributed Stochastic Neighbor Embedding (t-SNE): A tool for eco-physiological transcriptomic analysis. , 2019, Marine genomics.

[3]  Yutaka Matsuo,et al.  Graph-based Knowledge Tracing: Modeling Student Proficiency Using Graph Neural Network , 2019, 2019 IEEE/WIC/ACM International Conference on Web Intelligence (WI).

[4]  Hui Xiong,et al.  EKT: Exercise-Aware Knowledge Tracing for Student Performance Prediction , 2019, IEEE Transactions on Knowledge and Data Engineering.

[5]  Jian Li,et al.  NetSMF: Large-Scale Network Embedding as Sparse Matrix Factorization , 2019, WWW.

[6]  Huan Liu,et al.  Interactive Unknowns Recommendation in E-Learning Systems , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[7]  Penghe Chen,et al.  Prerequisite-Driven Deep Knowledge Tracing , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[8]  Jie Zhang,et al.  Semi-supervised Learning on Graphs with Generative Adversarial Nets , 2018, CIKM.

[9]  Dit-Yan Yeung,et al.  Addressing two problems in deep knowledge tracing via prediction-consistent regularization , 2018, L@S.

[10]  Linfeng Shi,et al.  Answering Multiple-Choice Questions in Geographical Gaokao with a Concept Graph , 2018, ESWC.

[11]  Penghe Chen,et al.  KnowEdu: A System to Construct Knowledge Graph for Education , 2018, IEEE Access.

[12]  Kevin Chen-Chuan Chang,et al.  Distance-Aware DAG Embedding for Proximity Search on Heterogeneous Graphs , 2018, AAAI.

[13]  Todd M. Gureckis,et al.  Knowledge Tracing Using the Brain , 2018, EDM.

[14]  Emmanuel Müller,et al.  VERSE: Versatile Graph Embeddings from Similarity Measures , 2018, WWW.

[15]  Radek Pelánek,et al.  Bayesian knowledge tracing, logistic models, and beyond: an overview of learner modeling techniques , 2017, User Modeling and User-Adapted Interaction.

[16]  Li Guo,et al.  Knowledge Graph Embedding with Iterative Guidance from Soft Rules , 2017, AAAI.

[17]  Jian Pei,et al.  A Survey on Network Embedding , 2017, IEEE Transactions on Knowledge and Data Engineering.

[18]  Le Wu,et al.  Tracking Knowledge Proficiency of Students with Educational Priors , 2017, CIKM.

[19]  Jian Li,et al.  Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec , 2017, WSDM.

[20]  Hno. Fabio Gallego ALL OUR CHILDREN LEARNING. A primer for parents, Teachers and other Educators , 2017 .

[21]  Kevin Chen-Chuan Chang,et al.  A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[22]  Matt P. Stevenson,et al.  A systematic review of concept mapping-based formative assessment processes in primary and secondary science education , 2017 .

[23]  Nitesh V. Chawla,et al.  metapath2vec: Scalable Representation Learning for Heterogeneous Networks , 2017, KDD.

[24]  Chengjiang Li,et al.  Prerequisite Relation Learning for Concepts in MOOCs , 2017, ACL.

[25]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[26]  Ran Liu,et al.  Towards reliable and valid measurement of individualized student parameters , 2017, EDM.

[27]  Palash Goyal,et al.  Graph Embedding Techniques, Applications, and Performance: A Survey , 2017, Knowl. Based Syst..

[28]  Enhong Chen,et al.  Question Difficulty Prediction for READING Problems in Standard Tests , 2017, AAAI.

[29]  Zhaohui Wu,et al.  Recovering Concept Prerequisite Relations from University Course Dependencies , 2017, AAAI.

[30]  Dit-Yan Yeung,et al.  Dynamic Key-Value Memory Networks for Knowledge Tracing , 2016, WWW.

[31]  Zhaohui Wu,et al.  Using Prerequisites to Extract Concept Maps fromTextbooks , 2016, CIKM.

[32]  Kevin Chen-Chuan Chang,et al.  Regularizing Structured Classifier with Conditional Probabilistic Constraints for Semi-supervised Learning , 2016, CIKM.

[33]  Mark D. McDonnell,et al.  Understanding Data Augmentation for Classification: When to Warp? , 2016, 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA).

[34]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[35]  A. Kaplan,et al.  Higher education and the digital revolution: About MOOCs, SPOCs, social media, and the Cookie Monster , 2016 .

[36]  Hongyu Guo,et al.  DAG-Structured Long Short-Term Memory for Semantic Compositionality , 2016, NAACL.

[37]  Ghassan Beydoun,et al.  Learning path adaptation in online learning systems , 2016, 2016 IEEE 20th International Conference on Computer Supported Cooperative Work in Design (CSCWD).

[38]  Eric P. Xing,et al.  Harnessing Deep Neural Networks with Logic Rules , 2016, ACL.

[39]  Qiongkai Xu,et al.  GraRep: Learning Graph Representations with Global Structural Information , 2015, CIKM.

[40]  Gang Wang,et al.  DAG-Recurrent Neural Networks for Scene Labeling , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Wenyi Huang,et al.  Measuring Prerequisite Relations Among Concepts , 2015, EMNLP.

[42]  Enhong Chen,et al.  Cognitive Modelling for Predicting Examinee Performance , 2015, IJCAI.

[43]  Leonidas J. Guibas,et al.  Deep Knowledge Tracing , 2015, NIPS.

[44]  Stephen H. Bach,et al.  Hinge-Loss Markov Random Fields and Probabilistic Soft Logic , 2015, J. Mach. Learn. Res..

[45]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[46]  Yiming Yang,et al.  Concept Graph Learning from Educational Data , 2015, WSDM.

[47]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[48]  Michael C. Mozer,et al.  Integrating latent-factor and knowledge-tracing models to predict individual differences in learning , 2014, EDM.

[49]  Jun Zhu,et al.  Robust RegBayes: Selectively Incorporating First-Order Logic Domain Knowledge into Bayesian Models , 2014, ICML.

[50]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[51]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[52]  Kenneth R. Koedinger,et al.  Individualized Bayesian Knowledge Tracing Models , 2013, AIED.

[53]  M. McHugh Interrater reliability: the kappa statistic , 2012, Biochemia medica.

[54]  Ming-Wei Chang,et al.  Structured learning with constrained conditional models , 2012, Machine Learning.

[55]  Marcelo Milrad,et al.  An innovative concept map approach for improving students' learning performance with an instant feedback mechanism , 2012, Br. J. Educ. Technol..

[56]  Zachary A. Pardos,et al.  KT-IDEM: introducing item difficulty to the knowledge tracing model , 2011, UMAP'11.

[57]  R. Bennett Formative assessment: a critical review , 2011 .

[58]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[59]  Ben Taskar,et al.  Posterior Regularization for Structured Latent Variable Models , 2010, J. Mach. Learn. Res..

[60]  Gideon S. Mann,et al.  Generalized Expectation Criteria for Semi-Supervised Learning with Weakly Labeled Data , 2010, J. Mach. Learn. Res..

[61]  Neil T. Heffernan,et al.  Addressing the assessment challenge with an online system that tutors as it assesses , 2009, User Modeling and User-Adapted Interaction.

[62]  Lauren Cifuentes,et al.  The comparative effect of individually-constructed vs. collaboratively-constructed computer-based concept maps , 2009, Comput. Educ..

[63]  Vincent Aleven,et al.  More Accurate Student Modeling through Contextual Estimation of Slip and Guess Probabilities in Bayesian Knowledge Tracing , 2008, Intelligent Tutoring Systems.

[64]  Kinshuk,et al.  Mining e-Learning Domain Concept Map from Academic Articles , 2006, Sixth IEEE International Conference on Advanced Learning Technologies (ICALT'06).

[65]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[66]  Ricardo Conejo,et al.  Introducing Prerequisite Relations in a Multi-layered Bayesian Student Model , 2005, User Modeling.

[67]  Pierre Baldi,et al.  The Principled Design of Large-Scale Recursive Neural Network Architectures--DAG-RNNs and the Protein Structure Prediction Problem , 2003, J. Mach. Learn. Res..

[68]  Jean-Claude Falmagne,et al.  Spaces for the Assessment of Knowledge , 1985, Int. J. Man Mach. Stud..

[69]  Eda Okur,et al.  Human Expert Labeling Process (HELP): Towards a Reliable Higher-order User State Labeling Process and Tool to Assess Student Engagement , 2017 .

[70]  C A Nelson,et al.  Learning to Learn , 2017, Encyclopedia of Machine Learning and Data Mining.

[71]  Yiming Yang,et al.  Data-driven Automated Induction of Prerequisite Structure Graphs , 2016, EDM.

[72]  Laxmisha Rai,et al.  Influencing Factors of Success and Failure in MOOC and General Analysis of Learner Behavior , 2016 .

[73]  Jin Tian,et al.  Joint Discovery of Skill Prerequisite Graphs and Student Models , 2016, EDM.

[74]  Yiming Yang,et al.  Learning Concept Graphs from Online Educational Data , 2016, J. Artif. Intell. Res..

[75]  Joseph E. Beck,et al.  Going Deeper with Deep Knowledge Tracing , 2016, EDM.

[76]  Sameer Singh,et al.  Injecting Logical Background Knowledge into Embeddings for Relation Extraction , 2015, NAACL.

[77]  James Gregory Jardine,et al.  Automatically generating reading lists , 2014 .

[78]  Zachary A. Pardos,et al.  Adapting Bayesian Knowledge Tracing to a Massive Open Online Course in edX , 2013, EDM.

[79]  Lars Schmidt-Thieme,et al.  Factorization Techniques for Predicting Student Performance , 2012 .

[80]  Neil T. Heffernan,et al.  Detecting Learning Moment-by-Moment , 2011, Int. J. Artif. Intell. Educ..

[81]  John R. Anderson,et al.  Knowledge tracing: Modeling the acquisition of procedural knowledge , 2005, User Modeling and User-Adapted Interaction.

[82]  J. Novak The Theory Underlying Concept Maps and How To Construct Them , 2004 .

[83]  Joseph D. Novak,et al.  Learning creating and using knowledge: Concept maps as facilitative tools , 1998 .