Dynamic models for knowledge tracing & prediction of future performance

Large-scale data about learners' behavior are being generated at high speed on various online learning platforms. Knowledge Tracing (KT) is a family of machine learning sequence models that are capable of using these data efficiently with the objective to identify the likelihood of future learning performance. This study provides an overview of KT models from a technical and an educational point of view. It focuses on data representation, evaluation, and optimization, and discusses the underlying model assumptions such that the strengths and weaknesses with regard to a specific application become visible. Based on the need for advanced analytical methods suited for large and diverse data, we briefly review big data analytics along with KT learning algorithms' efficiency, learnability and scalability. Challenges and future research directions are also outlined. In general, the overview can serve as a guide for researchers and developers, linking the dynamic knowledge tracing models and properties to the learner's knowledge acquisition process that should be accurately modeled over time. Applied KT models to online learning environments hold great potential for the online education industry because it enables the development of personalized adaptive learning systems. Keywords-big data applications; educational data mining; knowledge tracing; sequential supervised machine learning.

[1]  U. Goswami,et al.  The Future of Educational Neuroscience , 2010 .

[2]  Cynthia Breazeal,et al.  Affect and Inference in Bayesian Knowledge Tracing with a Robot Tutor , 2015, HRI.

[3]  Daniyal M. Alghazzawi,et al.  A Survey of Artificial Intelligence Techniques Employed for Adaptive Educational Systems within E-Learning Platforms , 2017, J. Artif. Intell. Soft Comput. Res..

[4]  Alejandro Peña Ayala,et al.  Educational data mining: A survey and a data mining-based analysis of recent works , 2014, Expert Syst. Appl..

[5]  Katrina Sin,et al.  Application of Big Data in Education Data Mining and Learning Analytics-A Literature Review , 2015, SOCO 2015.

[6]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[7]  Vincent Aleven,et al.  More Accurate Student Modeling through Contextual Estimation of Slip and Guess Probabilities in Bayesian Knowledge Tracing , 2008, Intelligent Tutoring Systems.

[8]  Ben Kei Daniel,et al.  Big Data and data science: A critical review of issues for educational research , 2019, Br. J. Educ. Technol..

[9]  Radek Pelanek,et al.  Applications of the Elo rating system in adaptive educational systems , 2016, Comput. Educ..

[10]  Yun Huang,et al.  Your Model Is Predictive - but Is It Useful? Theoretical and Empirical Considerations of a New Paradigm for Adaptive Tutoring Evaluation , 2015, EDM.

[11]  Kenneth R. Koedinger,et al.  Individualized Bayesian Knowledge Tracing Models , 2013, AIED.

[12]  A. Elliot,et al.  Handbook of Competence and Motivation , 2013 .

[13]  Radek Pelánek,et al.  Bayesian knowledge tracing, logistic models, and beyond: an overview of learner modeling techniques , 2017, User Modeling and User-Adapted Interaction.

[14]  Sebastián Ventura,et al.  Educational data science in massive open online courses , 2016, WIREs Data Mining Knowl. Discov..

[15]  Dit-Yan Yeung,et al.  Addressing two problems in deep knowledge tracing via prediction-consistent regularization , 2018, L@S.

[16]  Chaitanya Ekanadham,et al.  T-SKIRT: Online Estimation of Student Proficiency in an Adaptive Learning System , 2017, ArXiv.

[17]  Zachary A. Pardos,et al.  Modeling Individualization in a Bayesian Networks Implementation of Knowledge Tracing , 2010, UMAP.

[18]  Alexander G. Schwing,et al.  Dynamic Bayesian Networks for Student Modeling , 2017, IEEE Transactions on Learning Technologies.

[19]  Sebastián Ventura,et al.  Educational Data Mining: A Review of the State of the Art , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[20]  Joseph E. Beck,et al.  Identifiability: A Fundamental Problem of Student Modeling , 2007, User Modeling.

[21]  Zachary A Pardos,et al.  Big data in education and the models that love them , 2017, Current Opinion in Behavioral Sciences.

[22]  Elizabeth A. Linnenbrink,et al.  Role of Affect in Cognitive Processing in Academic Contexts , 2004 .

[23]  Michael C. Mozer,et al.  How Deep is Knowledge Tracing? , 2016, EDM.

[24]  Radek Pelánek,et al.  Metrics for Evaluation of Student Models , 2015, EDM.

[25]  John R. Anderson,et al.  Knowledge tracing: Modeling the acquisition of procedural knowledge , 2005, User Modeling and User-Adapted Interaction.

[26]  Robert V. Lindsey,et al.  Incorporating Latent Factors Into Knowledge Tracing To Predict Individual Differences In Learning , 2013 .

[27]  Neil T. Heffernan,et al.  Comparing Knowledge Tracing and Performance Factor Analysis by Using Multiple Model Fitting Procedures , 2010, Intelligent Tutoring Systems.

[28]  Zhi-Ting Zhu,et al.  A research framework of smart education , 2016, Smart Learning Environments.

[29]  Zachary A. Pardos,et al.  Navigating the parameter space of Bayesian Knowledge Tracing models: Visualizations of the convergence of the Expectation Maximization algorithm , 2010, EDM.

[30]  Sebastián Ventura,et al.  A Survey on Pre-Processing Educational Data , 2014 .

[31]  Paul Prinsloo,et al.  Big(ger) Data as Better Data in Open Distance Learning. , 2015 .

[32]  David Gibson,et al.  Big Data in Higher Education: Research Methods and Analytics Supporting the Learning Journey , 2017, Technology, Knowledge and Learning.

[33]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Alfred Essa A possible future for next generation adaptive learning systems , 2016, Smart Learning Environments.

[35]  Pedro M. Domingos A few useful things to know about machine learning , 2012, Commun. ACM.

[36]  Chaitanya Ekanadham,et al.  Back to the basics: Bayesian extensions of IRT outperform neural networks for proficiency estimation , 2016, EDM.

[37]  Rachida Dssouli,et al.  Big Data Pre-processing: A Quality Framework , 2015, 2015 IEEE International Congress on Big Data.

[38]  Geoffrey L. Cohen,et al.  Identity, Belonging, and Achievement , 2008 .

[39]  Anastasios A. Economides,et al.  Learning Analytics and Educational Data Mining in Practice: A Systematic Literature Review of Empirical Evidence , 2014, J. Educ. Technol. Soc..

[40]  B. J. Fogg,et al.  A behavior model for persuasive design , 2009, Persuasive '09.

[41]  Peter Brusilovsky,et al.  Integrating Knowledge Tracing and Item Response Theory: A Tale of Two Frameworks , 2014, UMAP Workshops.

[42]  Vincent Aleven,et al.  Modeling Students' Metacognitive Errors in Two Intelligent Tutoring Systems , 2005, User Modeling.

[43]  Emma Brunskill,et al.  The Impact on Individualizing Student Models on Necessary Practice Opportunities , 2012, EDM.