Refocusing the lens on engagement in MOOCs

Massive open online courses (MOOCs) continue to see increasing enrollment and adoption by universities, although they are still not fully understood and could perhaps be significantly improved. For example, little is known about the relationships between the ways in which students choose to use MOOCs (e.g., sampling lecture videos, discussing topics with fellow students) and their overall level of engagement with the course, although these relationships are likely key to effective course implementation. In this paper we propose a multilevel definition of student engagement with MOOCs and explore the connections between engagement and students' behaviors across five unique courses. We modeled engagement using ordinal penalized logistic regression with the least absolute shrinkage and selection operator (LASSO), and found several predictors of engagement that were consistent across courses. In particular, we found that discussion activities (e.g., viewing forum posts) were positively related to engagement, whereas other types of student behaviors (e.g., attempting quizzes) were consistently related to less engagement with the course. Finally, we discuss implications of unexpected findings that replicated across courses, future work to explore these implications, and relevance of our findings for MOOC course design.

[1]  Carolyn Penstein Rosé,et al.  “ Turn on , Tune in , Drop out ” : Anticipating student dropouts in Massive Open Online Courses , 2013 .

[2]  James Bailey,et al.  Identifying At-Risk Students in Massive Open Online Courses , 2015, AAAI.

[3]  Qian Zhang,et al.  Modeling and Predicting Learning Behavior in MOOCs , 2016, WSDM.

[4]  Danielle S. McNamara,et al.  Predicting Success in Massive Open Online Courses (MOOCs) Using Cohesion Network Analysis , 2017, CSCL.

[5]  A. Agresti Categorical data analysis , 1993 .

[6]  Sherif Halawa,et al.  Attrition and Achievement Gaps in Online Learning , 2015, L@S.

[7]  Justin Reich,et al.  Forecasting student achievement in MOOCs with natural language processing , 2016, LAK.

[8]  Andrew D. Ho,et al.  Changing “Course” , 2014 .

[9]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[10]  Hugh C. Davis,et al.  How Learners’ Interactions Sustain Engagement: A MOOC Case Study , 2017, IEEE Transactions on Learning Technologies.

[11]  Niels Pinkwart,et al.  Predicting MOOC Dropout over Weeks Using Machine Learning Methods , 2014, EMNLP 2014.

[12]  Paula de Barba,et al.  The role of students' motivation and participation in predicting performance in a MOOC , 2016, J. Comput. Assist. Learn..

[13]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[14]  Steven D'Alessandro,et al.  Learner motivation for MOOC registration and the role of MOOCs as a university ‘taster’ , 2016 .

[15]  Katharina Reinecke,et al.  Demographic differences in how students navigate through MOOCs , 2014, L@S.

[16]  George Siemens,et al.  Replicating 21 findings on student success in online learning , 2017 .

[17]  Stephanie D. Teasley,et al.  Who You Are or What You Do: Comparing the Predictive Power of Demographics vs. Activity Patterns in Massive Open Online Courses (MOOCs) , 2015, L@S.

[18]  Lu Ding,et al.  Studies of student engagement in gamified online discussions , 2017, Comput. Educ..

[19]  Mark Warschauer,et al.  Predicting MOOC performance with Week 1 Behavior , 2014, EDM.

[20]  J. Greene,et al.  Predictors of Retention and Achievement in a Massive Open Online Course , 2015 .

[21]  Jacob Whitehill,et al.  Delving Deeper into MOOC Student Dropout Prediction , 2017, ArXiv.

[22]  Doug Clow,et al.  MOOCs and the funnel of participation , 2013, LAK '13.

[23]  Danielle S. McNamara,et al.  Combining click-stream data with NLP tools to better understand MOOC completion , 2016, LAK.

[24]  Carolyn J. Anderson,et al.  How do Gender, Learning Goals, and Forum Participation Predict Persistence in a Computer Science MOOC? , 2018, ACM Trans. Comput. Educ..

[25]  Sherif A. Halawa,et al.  Dropout Prediction in MOOCs using Learner Activity Features , 2014 .

[26]  Chris Piech,et al.  Deconstructing disengagement: analyzing learner subpopulations in massive open online courses , 2013, LAK '13.

[27]  Patrick C. Shih,et al.  Understanding Student Motivation, Behaviors and Perceptions in MOOCs , 2015, CSCW.

[28]  Paul J. Rathouz,et al.  Regularized Ordinal Regression and the ordinalNet R Package , 2017, J. Stat. Softw..

[29]  Arthur E. Hoerl,et al.  Ridge Regression: Biased Estimation for Nonorthogonal Problems , 2000, Technometrics.

[30]  Steve Joordens,et al.  Assessing the effectiveness of a voluntary online discussion forum on improving students' course performance , 2011, Comput. Educ..

[31]  Trevor Hastie,et al.  Statistical Learning with Sparsity: The Lasso and Generalizations , 2015 .

[32]  Justin Reich,et al.  Staggered Versus All-At-Once Content Release in Massive Open Online Courses: Evaluating a Natural Experiment , 2015, L@S.

[33]  Lise Getoor,et al.  Learning Latent Engagement Patterns of Students in Online Courses , 2014, AAAI.

[34]  Carolyn Penstein Rosé,et al.  Linguistic Reflections of Student Engagement in Massive Open Online Courses , 2014, ICWSM.

[35]  Jure Leskovec,et al.  Engaging with massive online courses , 2014, WWW.