Click-Based Student Performance Prediction: A Clustering Guided Meta-Learning Approach

We study the problem of predicting student knowledge acquisition in online courses from clickstream behavior. Motivated by the proliferation of eLearning lecture delivery, we specifically focus on student in-video activity in lectures videos, which consist of content and in-video quizzes. Our methodology for predicting in-video quiz performance is based on three key ideas we develop. First, we model students’ clicking behavior via time-series learning architectures operating on raw event data, rather than defining hand-crafted features as in existing approaches that may lose important information embedded within the click sequences. Second, we develop a self-supervised clickstream pre-training to learn informative representations of clickstream events that can initialize the prediction model effectively. Third, we propose a clustering guided meta-learning-based training that optimizes the prediction model to exploit clusters of frequent patterns in student clickstream sequences. Through experiments on three real-world datasets, we demonstrate that our method obtains substantial improvements over two base-line models in predicting students’ in-video quiz performance. Further, we validate the importance of the pre-training and meta-learning components of our framework through ablation studies. Finally, we show how our methodology reveals insights on video-watching behavior associated with knowledge acquisition for useful learning analytics.

[1]  Mung Chiang,et al.  Learner Behavioral Feature Refinement and Augmentation Using GANs , 2018, AIED.

[2]  Olasile Babatunde Adedoyin,et al.  Covid-19 pandemic and online learning: the challenges and opportunities , 2020, Interact. Learn. Environ..

[3]  Yong Luo,et al.  A MOOC Video Viewing Behavior Analysis Algorithm , 2018 .

[4]  Prateek Mittal,et al.  Learning Informative and Private Representations via Generative Adversarial Networks , 2018, 2018 IEEE International Conference on Big Data (Big Data).

[5]  Niels Pinkwart,et al.  Predicting MOOC Dropout over Weeks Using Machine Learning Methods , 2014, EMNLP 2014.

[6]  Patrick Jermann,et al.  Your click decides your fate: Inferring Information Processing and Attrition Behavior from MOOC Video Clickstream Interactions , 2014, Proceedings of the EMNLP 2014 Workshop on Analysis of Large Scale Social Interaction in MOOCs.

[7]  Jungpin Wu,et al.  PREDICTING LEARNING OUTCOMES WITH MOOCS CLICKSTREAMS , 2019, Educational Innovations and Applications.

[8]  Mung Chiang,et al.  MOOC performance prediction via clickstream data and social learning networks , 2015, 2015 IEEE Conference on Computer Communications (INFOCOM).

[9]  Mung Chiang,et al.  Behavioral Analysis at Scale: Learning Course Prerequisite Structures from Learner Clickstreams , 2018, EDM.

[10]  Carolyn Penstein Rosé,et al.  Exploring the Effect of Student Confusion in Massive Open Online Courses , 2016, EDM.

[11]  Mung Chiang,et al.  Personalized Thread Recommendation for MOOC Discussion Forums , 2018, ECML/PKDD.

[12]  Abhinav Dhall,et al.  EmotiW 2020: Driver Gaze, Group Emotion, Student Engagement and Physiological Signal based Challenges , 2020, ICMI.

[13]  Mustafa Said Kıymaz,et al.  Understanding the Most Important Facilitators and Barriers for Online Education during COVID-19 through Online Photovoice Methodology , 2020 .

[14]  Daniel M. Russell,et al.  Student skill and goal achievement in the mapping with google MOOC , 2014, L@S.

[15]  Armando M. Toda,et al.  Social Interactions Clustering MOOC Students: An Exploratory Study , 2020, ArXiv.

[16]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Daphne Koller,et al.  Self-Paced Learning for Latent Variable Models , 2010, NIPS.

[18]  Qi Xie,et al.  Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting , 2019, NeurIPS.

[19]  Chunyan Miao,et al.  Deep Model for Dropout Prediction in MOOCs , 2017, ICCSE'17.

[20]  Byungsoo Jeon,et al.  Dropout Prediction over Weeks in MOOCs via Interpretable Multi-Layer Representation Learning , 2020, ArXiv.

[21]  Feng Zhang,et al.  MOOC Video Personalized Classification Based on Cluster Analysis and Process Mining , 2020, Sustainability.

[22]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[23]  Sherif A. Halawa,et al.  Dropout Prediction in MOOCs using Learner Activity Features , 2014 .

[24]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[25]  Andrew S. Lan,et al.  BOBCAT: Bilevel Optimization-Based Computerized Adaptive Testing , 2021, IJCAI.

[26]  Zichao Wang,et al.  A Meta-Learning Augmented Bidirectional Transformer Model for Automatic Short Answer Grading , 2019, EDM.

[27]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[28]  H. Vincent Poor,et al.  Mining MOOC Clickstreams: Video-Watching Behavior vs. In-Video Quiz Performance , 2016, IEEE Transactions on Signal Processing.

[29]  Byungsoo Jeon,et al.  Dropout Prediction over Weeks in MOOCs by Learning Representations of Clicks and Videos , 2020, ArXiv.

[30]  Joseph Jay Williams,et al.  Beyond Prediction: Towards Automatic Intervention in MOOC Student Stop-out , 2015, EDM.

[31]  Mina Shirvani Boroujeni,et al.  Discovery and temporal analysis of latent study patterns in MOOC interaction sequences , 2018, LAK.

[32]  Hang Li,et al.  Convolutional Neural Network Architectures for Matching Natural Language Sentences , 2014, NIPS.

[33]  Louise F. Pendry,et al.  Individual and social benefits of online discussion forums , 2015, Comput. Hum. Behav..

[34]  Nitesh V. Chawla,et al.  MOOC Dropout Prediction: Lessons Learned from Making Pipelines Interpretable , 2017, WWW.

[35]  Tolga Güyer,et al.  Students’ interaction patterns in different online learning activities and their relationship with motivation, self-regulated learning strategy and learning performance , 2020, Education and Information Technologies.

[36]  Michael I. Jordan,et al.  Machine learning: Trends, perspectives, and prospects , 2015, Science.

[37]  Dit-Yan Yeung,et al.  Dynamic Key-Value Memory Networks for Knowledge Tracing , 2016, WWW.

[38]  Collin Lynch,et al.  Your Actions or Your Associates? Predicting Certification and Dropout in MOOCs with Behavioral and Social Features , 2018, EDM.

[39]  Javier R. Movellan,et al.  The Faces of Engagement: Automatic Recognition of Student Engagementfrom Facial Expressions , 2014, IEEE Transactions on Affective Computing.

[40]  Zhenming Liu,et al.  Learning about Social Learning in MOOCs: From Statistical Analysis to Generative Model , 2013, IEEE Transactions on Learning Technologies.

[41]  Taylor V. Williams,et al.  Characterizing MOOC Learners from Survey Data Using Modeling and n-TARP Clustering , 2018 .

[42]  Kalyan Veeramachaneni,et al.  Likely to stop? Predicting Stopout in Massive Open Online Courses , 2014, ArXiv.

[43]  Mung Chiang,et al.  Behavior-Based Latent Variable Model for Learner Engagement , 2017, EDM.

[44]  Mung Chiang,et al.  Behavior-Based Grade Prediction for MOOCs Via Time Series Neural Networks , 2017, IEEE Journal of Selected Topics in Signal Processing.

[45]  Fred G. Martin,et al.  Will massive open online courses change how we teach? , 2012, CACM.

[46]  Christopher G. Brinton,et al.  Predictive learning analytics for video-watching behavior in MOOCs , 2018, 2018 52nd Annual Conference on Information Sciences and Systems (CISS).

[47]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[48]  H. Vincent Poor,et al.  On the Efficiency of Online Social Learning Networks , 2018, IEEE/ACM Transactions on Networking.

[49]  Michael Jahrer,et al.  Collaborative Filtering Applied to Educational Data Mining , 2010 .

[50]  Katharina Reinecke,et al.  Demographic differences in how students navigate through MOOCs , 2014, L@S.

[51]  Alexei A. Efros,et al.  Ensemble of exemplar-SVMs for object detection and beyond , 2011, 2011 International Conference on Computer Vision.

[52]  Dit-Yan Yeung,et al.  Temporal Models for Predicting Student Dropout in Massive Open Online Courses , 2015, 2015 IEEE International Conference on Data Mining Workshop (ICDMW).

[53]  Heng Luo,et al.  Learning Profiles, Behaviors and Outcomes: Investigating International Students' Learning Experience in an English MOOC , 2018, 2018 International Symposium on Educational Technology (ISET).

[54]  Chinchu Thomas Multimodal Teaching and Learning Analytics for Classroom and Online Educational Settings , 2018, ICMI.

[55]  Dit-Yan Yeung,et al.  Clickstream Knowledge Tracing: Modeling How Students Answer Interactive Online Questions , 2021, LAK.