Temporal analysis for dropout prediction using self-regulated learning strategies in self-paced MOOCs

Abstract MOOCs (Massive Open Online Courses) have usually high dropout rates. Many articles have proposed predictive models in order to early detect learners at risk to alleviate this issue. Nevertheless, existing models do not consider complex high-level variables, such as self-regulated learning (SRL) strategies, which can have an important effect on learners' success. In addition, predictions are often carried out in instructor-paced MOOCs, where contents are released gradually, but not in self-paced MOOCs, where all materials are available from the beginning and users can enroll at any time. For self-paced MOOCs, existing predictive models are limited in the way they deal with the flexibility offered by the course start date, which is learner dependent. Therefore, they need to be adapted so as to predict with little information short after each learner starts engaging with the MOOC. To solve these issues, this paper contributes with the study of how SRL strategies could be included in predictive models for self-paced MOOCs. Particularly, self-reported and event-based SRL strategies are evaluated and compared to measure their effect for dropout prediction. Also, the paper contributes with a new methodology to analyze self-paced MOOCs when carrying out a temporal analysis to discover how early prediction models can serve to detect learners at risk. Results of this article show that event-based SRL strategies show a very high predictive power, although variables related to learners' interactions with exercises are still the best predictors. That is, event-based SRL strategies can be useful to predict if e.g., variables related to learners' interactions with exercises are not available. Furthermore, results show that this methodology serves to achieve early powerful predictions from about 25 to 33% of the theoretical course duration. The proposed methodology presents a new approach to predict dropouts in self-paced MOOCs, considering complex variables that go beyond the classic trace-data directly captured by the MOOC platforms.

[1]  Mark Warschauer,et al.  Predicting MOOC performance with Week 1 Behavior , 2014, EDM.

[2]  A Bandura,et al.  Comments on the crusade against the causal efficacy of human thought. , 1995, Journal of behavior therapy and experimental psychiatry.

[3]  Hiroaki Ogata,et al.  A neural network approach for students' performance prediction , 2017, LAK.

[4]  Mung Chiang,et al.  MOOC performance prediction via clickstream data and social learning networks , 2015, 2015 IEEE Conference on Computer Communications (INFOCOM).

[5]  Gilles Louppe,et al.  Understanding variable importances in forests of randomized trees , 2013, NIPS.

[6]  Patrick C. Shih,et al.  Understanding Student Motivation, Behaviors and Perceptions in MOOCs , 2015, CSCW.

[7]  Jie Tang,et al.  Understanding Dropouts in MOOCs , 2019, AAAI.

[8]  Olivier Le Bohec,et al.  Procrastination, participation, and performance in online learning environments , 2011, Comput. Educ..

[9]  Katrien Verbert,et al.  Generalizing Predictive Models of Admission Test Success Based on Online Interactions , 2019, Sustainability.

[10]  Nicolás Morales,et al.  Mining theory-based patterns from Big data: Identifying self-regulated learning strategies in Massive Open Online Courses , 2018, Comput. Hum. Behav..

[11]  Radek Pelánek,et al.  Metrics for Evaluation of Student Models , 2015, EDM.

[12]  M. Boekaerts.  SELF-REGULATED LEARNING: A NEW CONCEPT EMBRACED BY RESEARCHERS, POLICY MAKERS, EDUCATORS, TEACHERS, AND STUDENTS , 1997 .

[13]  Ali Shiri,et al.  Predictive analytic models of student success in higher education , 2019, Information and Learning Sciences.

[14]  Christian Gütl,et al.  User Behavioral Patterns and Early Dropouts Detection: Improved Users Profiling through Analysis of Successive Offering of MOOC , 2018, J. Univers. Comput. Sci..

[15]  Carlos Delgado Kloos,et al.  Prediction in MOOCs: A Review and Future Research Directions , 2019, IEEE Transactions on Learning Technologies.

[16]  Sherif A. Halawa,et al.  Dropout Prediction in MOOCs using Learner Activity Features , 2014 .

[17]  Mar Pérez-Sanagustín,et al.  Self-regulated learning strategies predict learner behavior and goal attainment in Massive Open Online Courses , 2017, Comput. Educ..

[18]  Kui Xie,et al.  The role of self-regulated learning in students' success in flipped undergraduate math courses , 2018, Internet High. Educ..

[19]  Carlos Delgado Kloos,et al.  Early Prediction and Variable Importance of Certificate Accomplishment in a MOOC , 2017, EMOOCs.

[20]  Felienne Hermans,et al.  Teaching Software Engineering Principles to K-12 Students: A MOOC on Scratch , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering: Software Engineering Education and Training Track (ICSE-SEET).

[21]  Fred Paas,et al.  Supporting Self-Regulated Learning in Online Learning Environments and MOOCs: A Systematic Review , 2018, Int. J. Hum. Comput. Interact..

[22]  J. Greene,et al.  Predictors of Retention and Achievement in a Massive Open Online Course , 2015 .

[23]  J. Daniel,et al.  Making Sense of MOOCs : Musings in a Maze of Myth , Paradox and Possibility Author : , 2013 .

[24]  Niels Pinkwart,et al.  Predicting MOOC Dropout over Weeks Using Machine Learning Methods , 2014, EMNLP 2014.

[25]  Allison Littlejohn,et al.  Context counts: How learners' contexts influence learning in a MOOC , 2015, Comput. Educ..

[26]  Jason Rhode,et al.  Interaction Equivalency in Self-Paced Online Learning Environments: An Exploration of Learner Preferences , 2009 .

[27]  Sunnie Lee Watson,et al.  Systematic literature review on self-regulated learning in massive open online courses , 2019, Australasian Journal of Educational Technology.

[28]  Hakan Altinpulluk,et al.  A Theoretical Analysis of Moocs Types from a Perspective of Learning Theories , 2015 .

[29]  Eduardo Gómez-Sánchez,et al.  Predicting the decrease of engagement indicators in a MOOC , 2017, LAK.

[30]  Anastasios A. Economides,et al.  Explaining learning performance using response-time, self-regulation and satisfaction from content: an fsQCA approach , 2018, LAK.

[31]  Carlos Delgado Kloos,et al.  Analysing the predictive power for anticipating assignment grades in a massive open online course , 2018, Behav. Inf. Technol..

[32]  Xin Chen,et al.  Temporal predication of dropouts in MOOCs: Reaching the low hanging fruit through stacking generalization , 2016, Comput. Hum. Behav..

[33]  S. Järvelä,et al.  Third wave of measurement in the self-regulated learning field: when measurement and intervention come hand in hand , 2016 .

[34]  C. Abraham,et al.  Psychological correlates of university students' academic performance: a systematic review and meta-analysis. , 2012, Psychological bulletin.

[35]  Youngju Lee,et al.  Discriminating factors between completers of and dropouts from online learning courses , 2013, Br. J. Educ. Technol..

[36]  Roger Azevedo,et al.  Teaching and Learning in Technology-Rich Environments , 2006 .

[37]  Deborah L Engle,et al.  Coursera's Introductory Human Physiology Course: Factors That Characterize Successful Completion of a MOOC. , 2015 .

[38]  Judith Ramsay,et al.  Massive open online courses (MOOCs): Insights and challenges from a psychological perspective , 2015, Br. J. Educ. Technol..

[39]  Fernando De la Torre,et al.  Facing Imbalanced Data--Recommendations for the Use of Performance Metrics , 2013, 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction.

[40]  Carlos Delgado Kloos,et al.  Predicting Learners' Success in a Self-paced MOOC Through Sequence Patterns of Self-regulated Learning , 2018, EC-TEL.

[41]  Dirk T. Tempelaar,et al.  Investigating learning strategies in a dispositional learning analytics context: the case of worked examples , 2018, LAK.

[42]  Amar-Djalil Mezaour,et al.  Filtering Web Documents for a Thematic Warehouse Case Study: eDot a Food Risk Data Warehouse (extended) , 2005, Intelligent Information Systems.

[43]  Kalyan Veeramachaneni,et al.  Transfer Learning for Predictive Models in Massive Open Online Courses , 2015, AIED.

[44]  René F. Kizilcec,et al.  Motivation as a Lens to Understand Online Learners , 2015, ACM Trans. Comput. Hum. Interact..

[45]  Wanli Xing,et al.  Dropout Prediction in MOOCs: Using Deep Learning for Personalized Intervention , 2019 .

[46]  Jaclyn Broadbent,et al.  Comparing online and blended learner's self-regulated learning strategies and academic performance , 2017, Internet High. Educ..

[47]  Gautam Biswas,et al.  Behavior Prediction in MOOCs using Higher Granularity Temporal Information , 2015, L@S.

[48]  Mar Pérez-Sanagustín,et al.  Recommending Self-Regulated Learning Strategies Does Not Improve Performance in a MOOC , 2016, L@S.

[49]  Scott D. Johnson,et al.  FACTORS THAT INFLUENCE STUDENTS’ DECISION TO DROPOUT OF ONLINE COURSES , 2019, Online Learning.

[50]  N. Sarat Chandra Babu,et al.  Implementation of learning analytics framework for MOOCs using state-of-the-art in-memory computing , 2017, 2017 5th National Conference on E-Learning & E-Learning Technologies (ELELTECH).

[51]  Stephanie D. Teasley,et al.  Who You Are or What You Do: Comparing the Predictive Power of Demographics vs. Activity Patterns in Massive Open Online Courses (MOOCs) , 2015, L@S.