Dynamic Knowledge Tracing Models for Large-Scale Adaptive Learning Environments

— Large-scale data about learners’ behavior are being generated at high speed on various online learning platforms. Knowledge Tracing (KT) is a family of machine learning sequence models that use these data to identify the likelihood of future learning performance. KT models hold great potential for the online education industry by enabling the development of personalized adaptive learning systems. This study provides an overview of five KT models from both a technical and an educational point of view. Each model is chosen based on the inclusion of at least one adaptive learning property. These are the recency effects of engagement with the learning resources, dynamic sequences of learning resources, inclusion of students’ differences, and learning resources dependencies. Furthermore, the study outlines for each model, the data representation, evaluation, and optimization component, together with their advantages and potential pitfalls. The aforementioned dimensions and the underlying model assumptions reveal potential strengths and weaknesses of each model with regard to a specific application. Based on the need for advanced analytical methods suited for large-scale data, we briefly review big data analytics along with KT learning algorithms’ scalability. Challenges and future research directions regarding learners’ performance prediction are outlined. The provided overview is intended to serve as a guide for researchers and system developers, linking the models to the learner’s knowledge acquisition process modeled over time.

[1]  Sandjai Bhulai,et al.  Dynamic models for knowledge tracing & prediction of future performance , 2018 .

[2]  Ilja Cornelisz,et al.  Student engagement with computerized practising: Ability, task value, and difficulty perceptions , 2018, J. Comput. Assist. Learn..

[3]  Dit-Yan Yeung,et al.  Addressing two problems in deep knowledge tracing via prediction-consistent regularization , 2018, L@S.

[4]  Nienke M. Ruijs,et al.  Delaying access to a problem-skipping option increases effortful practice: Application of an A/B test in large-scale online learning , 2018, Comput. Educ..

[5]  Gary Marcus,et al.  Deep Learning: A Critical Appraisal , 2018, ArXiv.

[6]  Radek Pelánek,et al.  Bayesian knowledge tracing, logistic models, and beyond: an overview of learner modeling techniques , 2017, User Modeling and User-Adapted Interaction.

[7]  Zachary A Pardos,et al.  Big data in education and the models that love them , 2017, Current Opinion in Behavioral Sciences.

[8]  Alexander G. Schwing,et al.  Dynamic Bayesian Networks for Student Modeling , 2017, IEEE Transactions on Learning Technologies.

[9]  David Gibson,et al.  Big Data in Higher Education: Research Methods and Analytics Supporting the Learning Journey , 2017, Technology, Knowledge and Learning.

[10]  Radek Pelánek,et al.  Experimental Analysis of Mastery Learning Criteria , 2017, UMAP.

[11]  Scott Lundberg,et al.  A Unified Approach to Interpreting Model Predictions , 2017, NIPS.

[12]  Geert-Jan Houben,et al.  Follow the successful crowd: raising MOOC completion rates through social comparison at scale , 2017, LAK.

[13]  Chaitanya Ekanadham,et al.  T-SKIRT: Online Estimation of Student Proficiency in an Adaptive Learning System , 2017, ArXiv.

[14]  Dit-Yan Yeung,et al.  Dynamic Key-Value Memory Networks for Knowledge Tracing , 2016, WWW.

[15]  Alfred Essa A possible future for next generation adaptive learning systems , 2016, Smart Learning Environments.

[16]  Sebastián Ventura,et al.  Educational data science in massive open online courses , 2016, WIREs Data Mining Knowl. Discov..

[17]  Radek Pelanek,et al.  Applications of the Elo rating system in adaptive educational systems , 2016, Comput. Educ..

[18]  Chaitanya Ekanadham,et al.  Back to the basics: Bayesian extensions of IRT outperform neural networks for proficiency estimation , 2016, EDM.

[19]  Zhi-ting Zhu,et al.  A research framework of smart education , 2016, Smart Learning Environments.

[20]  Zhi-Ting Zhu,et al.  A research framework of smart education , 2016, Smart Learning Environments.

[21]  Richard S. Zemel,et al.  Gated Graph Sequence Neural Networks , 2015, ICLR.

[22]  Katrina Sin,et al.  Application of Big Data in Education Data Mining and Learning Analytics-A Literature Review , 2015, SOCO 2015.

[23]  Rachida Dssouli,et al.  Big Data Pre-processing: A Quality Framework , 2015, 2015 IEEE International Congress on Big Data.

[24]  Leonidas J. Guibas,et al.  Deep Knowledge Tracing , 2015, NIPS.

[25]  Radek Pelánek,et al.  Metrics for Evaluation of Student Models , 2015, EDM.

[26]  Yun Huang,et al.  Your Model Is Predictive - but Is It Useful? Theoretical and Empirical Considerations of a New Paradigm for Adaptive Tutoring Evaluation , 2015, EDM.

[27]  Cynthia Breazeal,et al.  Affect and Inference in Bayesian Knowledge Tracing with a Robot Tutor , 2015, HRI.

[28]  Paul Prinsloo,et al.  Big(ger) Data as Better Data in Open Distance Learning. , 2015 .

[29]  Anastasios A. Economides,et al.  Learning Analytics and Educational Data Mining in Practice: A Systematic Literature Review of Empirical Evidence , 2014, J. Educ. Technol. Soc..

[30]  Peter Brusilovsky,et al.  General Features in Knowledge Tracing to Model Multiple Subskills, Temporal Item Response Theory, and Expert Knowledge , 2014, EDM.

[31]  Alejandro Peña Ayala,et al.  Educational data mining: A survey and a data mining-based analysis of recent works , 2014, Expert Syst. Appl..

[32]  A. Elliot,et al.  Handbook of Competence and Motivation , 2013 .

[33]  Kenneth R. Koedinger,et al.  Individualized Bayesian Knowledge Tracing Models , 2013, AIED.

[34]  Pedro M. Domingos A few useful things to know about machine learning , 2012, Commun. ACM.

[35]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Emma Brunskill,et al.  The Impact on Individualizing Student Models on Necessary Practice Opportunities , 2012, EDM.

[37]  Michel C. Desmarais,et al.  A review of recent advances in learner and skill modeling in intelligent learning environments , 2012, User Modeling and User-Adapted Interaction.

[38]  Sebastián Ventura,et al.  Educational Data Mining: A Review of the State of the Art , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[39]  Zachary A. Pardos,et al.  Modeling Individualization in a Bayesian Networks Implementation of Knowledge Tracing , 2010, UMAP.

[40]  Neil T. Heffernan,et al.  Comparing Knowledge Tracing and Performance Factor Analysis by Using Multiple Model Fitting Procedures , 2010, Intelligent Tutoring Systems.

[41]  U. Goswami,et al.  The Future of Educational Neuroscience , 2010 .

[42]  B. J. Fogg,et al.  A behavior model for persuasive design , 2009, Persuasive '09.

[43]  Vincent Aleven,et al.  More Accurate Student Modeling through Contextual Estimation of Slip and Guess Probabilities in Bayesian Knowledge Tracing , 2008, Intelligent Tutoring Systems.

[44]  Joseph E. Beck,et al.  Identifiability: A Fundamental Problem of Student Modeling , 2007, User Modeling.

[45]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[46]  Vincent Aleven,et al.  Modeling Students' Metacognitive Errors in Two Intelligent Tutoring Systems , 2005, User Modeling.

[47]  Elizabeth A. Linnenbrink,et al.  Role of Affect in Cognitive Processing in Academic Contexts , 2004 .

[48]  Ben Kei Daniel,et al.  Big Data and data science: A critical review of issues for educational research , 2019, Br. J. Educ. Technol..

[49]  Daniyal M. Alghazzawi,et al.  A Survey of Artificial Intelligence Techniques Employed for Adaptive Educational Systems within E-Learning Platforms , 2017, J. Artif. Intell. Soft Comput. Res..

[50]  Robert V. Lindsey,et al.  Incorporating Latent Factors Into Knowledge Tracing To Predict Individual Differences In Learning , 2013 .

[51]  Zachary A. Pardos,et al.  Navigating the parameter space of Bayesian Knowledge Tracing models: Visualizations of the convergence of the Expectation Maximization algorithm , 2010, EDM.

[52]  Yoshua Bengio,et al.  Scaling learning algorithms towards AI , 2007 .

[53]  Peter Brusilovsky,et al.  User Models for Adaptive Hypermedia and Adaptive Educational Systems , 2007, The Adaptive Web.

[54]  Alfred Kobsa,et al.  The Adaptive Web, Methods and Strategies of Web Personalization , 2007, The Adaptive Web.

[55]  John R. Anderson,et al.  Knowledge tracing: Modeling the acquisition of procedural knowledge , 2005, User Modeling and User-Adapted Interaction.