Exploring the influences of MOOC design features on student performance and persistence

ABSTRACT Massive open online courses (MOOCs) face persistent challenges related to student performance, including high rates of attrition and low student achievement scores. Previous studies that have examined the performance of students in MOOCs have done so using qualitative analysis and the quantitative analysis of small samples. This study is the first to examine general course features of MOOCs on a large scale and to quantify the influences of these course features on student performance. Informed by the theory of web-based online instruction, this study used two-stage K-means clustering to analyze more than 200 MOOCs that had enrolled about 300,000 students, identifying three patterns of course features among the MOOCs. A MANOVA test and follow-up statistical tests revealed that these patterns of course features influenced the MOOCs’ dropout rates and student achievement scores to statistically different degrees. The implications of these findings are discussed.

[1]  William R. Watson,et al.  Instructor’s Use of Social Presence, Teaching Presence, and Attitudinal Dissonance: A Case Study of an Attitudinal Change MOOC , 2016 .

[2]  Armando Fox,et al.  Monitoring MOOCs: which information sources do instructors value? , 2014, L@S.

[3]  George Veletsianos,et al.  A Systematic Analysis and Synthesis of the Empirical MOOC Literature Published in 2013–2015 , 2016 .

[4]  Carolyn Penstein Rosé,et al.  “ Turn on , Tune in , Drop out ” : Anticipating student dropouts in Massive Open Online Courses , 2013 .

[5]  Mark Warschauer,et al.  Predicting MOOC performance with Week 1 Behavior , 2014, EDM.

[6]  Barbara L. Grabowski,et al.  Information, Instruction, and Learning: A Hypermedia Perspective. , 2008 .

[7]  Paula de Barba,et al.  The role of students' motivation and participation in predicting performance in a MOOC , 2016, J. Comput. Assist. Learn..

[8]  Martin Ebner,et al.  “How satisfied are you with your MOOC?” - A Research Study on Interaction in Huge Online Courses , 2013 .

[9]  Ulrik Schroeder,et al.  What Drives a Successful MOOC? An Empirical Examination of Criteria to Assure Design Quality of MOOCs , 2014, 2014 IEEE 14th International Conference on Advanced Learning Technologies.

[10]  Wilfried Admiraal,et al.  Self- and Peer Assessment in Massive Open Online Courses , 2014 .

[11]  Olaf Zawacki-Richter,et al.  Trends and Patterns in Massive Open Online Courses: Review and Content Analysis of Research on MOOCs (2008-2015) , 2017 .

[12]  Linda Corrin,et al.  Predicting success: how learners' prior knowledge, skills and activities predict MOOC performance , 2015, LAK.

[13]  M. David Merrill,et al.  First Principles of Instruction: Identifying and Designing Effective, Efficient, and Engaging Instruction , 2012 .

[14]  Papia Bawa,et al.  Retention in Online Courses , 2016 .

[15]  John R. Grandzol,et al.  Best Practices for Online Business Education , 2006 .

[16]  Allison Littlejohn,et al.  Instructional quality of Massive Open Online Courses (MOOCs) , 2015, Comput. Educ..

[17]  Sherif A. Halawa,et al.  Dropout Prediction in MOOCs using Learner Activity Features , 2014 .

[18]  Nigel Bosch,et al.  Refocusing the lens on engagement in MOOCs , 2018, L@S.

[19]  D. Garrison,et al.  Critical thinking, cognitive presence, and computer conferencing in distance education , 2001 .

[20]  M. David Merrill,et al.  First principles of instruction , 2012 .

[21]  Peggy A. Ertmer,et al.  Instructional design, facilitation, and perceived learning outcomes: an exploratory case study of a human trafficking MOOC for attitudinal change , 2016 .

[22]  Donatella Persico,et al.  Methodological approaches in MOOC research: Retracing the myth of Proteus , 2015, Br. J. Educ. Technol..

[23]  Tiffany A. Koszalka,et al.  Designing online courses: A taxonomy to guide strategic use of features available in course management systems (CMS) in distance education , 2004 .

[24]  Chris Piech,et al.  Deconstructing disengagement: analyzing learner subpopulations in massive open online courses , 2013, LAK '13.

[25]  Ulrike von Luxburg,et al.  Peer Grading in a Course on Algorithms and Data Structures: Machine Learning Algorithms do not Improve over Simple Baselines , 2016, L@S.

[26]  Olaf Zawacki-Richter,et al.  What Research Says About MOOCs – An Explorative Content Analysis , 2018 .

[27]  Shanna Smith Jaggars,et al.  Effectiveness of Fully Online Courses for College Students: Response to a Department of Education Meta-Analysis , 2010 .

[28]  Gautam Biswas,et al.  Behavior Prediction in MOOCs using Higher Granularity Temporal Information , 2015, L@S.

[29]  Geoffrey H. Ball,et al.  ISODATA, A NOVEL METHOD OF DATA ANALYSIS AND PATTERN CLASSIFICATION , 1965 .

[30]  Jinwoo Kim,et al.  Escape from infinite freedom: Effects of constraining user freedom on the prevention of dropout in an online learning context , 2017, Comput. Hum. Behav..

[31]  Geza Kovacs,et al.  Effects of In-Video Quizzes on MOOC Lecture Viewing , 2016, L@S.

[32]  Yvonne Belanger,et al.  Bioelectricity: A Quantitative Approach Duke University’s First MOOC , 2013 .

[33]  Barbara Means,et al.  Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies , 2009 .

[34]  Youngjun Lee,et al.  A review of online course dropout research: implications for practice and future research , 2011 .

[35]  Justin Reich,et al.  Forecasting student achievement in MOOCs with natural language processing , 2016, LAK.

[36]  Di Xu,et al.  How do online course design features influence student performance? , 2016, Comput. Educ..

[37]  Rui Guo,et al.  Identifying Time-of-Day Breakpoints Based on Nonintrusive Data Collection Platforms , 2014, J. Intell. Transp. Syst..

[38]  Patrick C. Shih,et al.  Understanding Student Motivation, Behaviors and Perceptions in MOOCs , 2015, CSCW.

[39]  Anil K. Jain.  Data clustering: 50 years beyond K-means , 2010, Pattern Recognit. Lett..

[40]  Mike Moore,et al.  Toward a Theory of Independent Learning and Teaching , 1973 .