TrueLearn: A Family of Bayesian Algorithms to Match Lifelong Learners to Open Educational Resources

The recent advances in computer-assisted learning systems and the availability of open educational resources today promise a pathway to providing cost-efficient high-quality education to large masses of learners. One of the most ambitious use cases of computer-assisted learning is to build a lifelong learning recommendation system. Unlike short-term courses, lifelong learning presents unique challenges, requiring sophisticated recommendation models that account for a wide range of factors such as background knowledge of learners or novelty of the material while effectively maintaining knowledge states of masses of learners for significantly longer periods of time (ideally, a lifetime). This work presents the foundations towards building a dynamic, scalable and transparent recommendation system for education, modelling learner's knowledge from implicit data in the form of engagement with open educational resources. We i) use a text ontology based on Wikipedia to automatically extract knowledge components of educational resources and, ii) propose a set of online Bayesian strategies inspired by the well-known areas of item response theory and knowledge tracing. Our proposal, TrueLearn, focuses on recommendations for which the learner has enough background knowledge (so they are able to understand and learn from the material), and the material has enough novelty that would help the learner improve their knowledge about the subject and keep them engaged. We further construct a large open educational video lectures dataset and test the performance of the proposed algorithms, which show clear promise towards building an effective educational recommendation system.

[1]  Radek Pelánek,et al.  Elo-based learner modeling for the adaptive practice of facts , 2017, User Modeling and User-Adapted Interaction.

[2]  Marko Grobelnik,et al.  Annotating documents with relevant Wikipedia concepts , 2017 .

[3]  Sebastián Ventura,et al.  An architecture for making recommendations to courseware authors using association rule mining and collaborative filtering , 2009, User Modeling and User-Adapted Interaction.

[4]  Hendrik Drachsler,et al.  Personal recommender systems for learners in lifelong learning networks: the requirements, techniques and model , 2008, Int. J. Learn. Technol..

[5]  John Shawe-Taylor,et al.  Towards Automatic, Scalable Quality Assurance in Open Education , 2019 .

[6]  Supreeth M. Gowda,et al.  Affective States and State Tests: Investigating How Affect and Engagement during the School Year Predict End-of-Year Learning Outcomes , 2014, J. Learn. Anal..

[7]  Mojtaba Salehi,et al.  Personalized recommendation of learning material using sequential pattern mining and attribute based collaborative filtering , 2012, Education and Information Technologies.

[8]  Kenneth R. Koedinger,et al.  Individualized Bayesian Knowledge Tracing Models , 2013, AIED.

[9]  Georg Rasch,et al.  Probabilistic Models for Some Intelligence and Attainment Tests , 1981, The SAGE Encyclopedia of Research Design.

[10]  John R. Anderson,et al.  Knowledge tracing: Modeling the acquisition of procedural knowledge , 2005, User Modeling and User-Adapted Interaction.

[11]  Dirk Van Damme,et al.  Open Educational Resources: Analysis of Responses to the OECD Country Questionnaire , 2012 .

[12]  Paul Covington,et al.  Deep Neural Networks for YouTube Recommendations , 2016, RecSys.

[13]  Tom Minka,et al.  TrueSkillTM: A Bayesian Skill Rating System , 2006, NIPS.

[14]  Sergey Brin,et al.  The Anatomy of a Large-Scale Hypertextual Web Search Engine , 1998, Comput. Networks.

[15]  Hisashi Kashima,et al.  Knowledge Tracing Machines: Factorization Machines for Knowledge Tracing , 2018, AAAI.

[16]  Douglas W. Oard,et al.  Implicit Feedback for Recommender Systems , 1998 .

[17]  Michael C. Mozer,et al.  Automatic Discovery of Cognitive Skills to Improve the Prediction of Student Learning , 2014, NIPS.

[18]  Judy Kay,et al.  SMILI☺: a Framework for Interfaces to Learning Data in Open Learner Models, Learning Analytics and Related Fields , 2016, International Journal of Artificial Intelligence in Education.

[19]  Albert T. Corbett,et al.  Cognitive Computer Tutors: Solving the Two-Sigma Problem , 2001, User Modeling.

[20]  Antonio Hernando,et al.  Collaborative filtering adapted to recommender systems of e-learning , 2009, Knowl. Based Syst..

[21]  Mung Chiang,et al.  Behavior-Based Latent Variable Model for Learner Engagement , 2017, EDM.

[22]  Zachary A. Pardos,et al.  Goal-based Course Recommendation , 2018, LAK.

[23]  Leonidas J. Guibas,et al.  Deep Knowledge Tracing , 2015, NIPS.

[24]  Richard G. Baraniuk,et al.  Time-varying learning and content analytics via sparse factor analysis , 2013, KDD.

[25]  Dietmar Jannach,et al.  Recommending Based on Implicit Feedback , 2018, Social Information Access.

[26]  Lise Getoor,et al.  Learning Latent Engagement Patterns of Students in Online Courses , 2014, AAAI.

[27]  John Shawe-Taylor,et al.  Towards an Integrative Educational Recommender for Lifelong Learners , 2019, AAAI.

[28]  Philip J. Guo,et al.  How video production affects student engagement: an empirical study of MOOC videos , 2014, L@S.

[29]  George D. Kuh,et al.  Student Engagement and Student Learning: Testing the Linkages* , 2006 .

[30]  Alexander Tuzhilin,et al.  Recommending Remedial Learning Materials to Students by Filling Their Knowledge Gaps , 2018, MIS Q..