A Randomized Experiment Testing the Efficacy of a Scheduling Nudge in a Massive Open Online Course (MOOC)

An increasing number of students are taking classes offered online through open-access platforms; however, the vast majority of students who start these classes do not finish. The incongruence of student intentions and subsequent engagement suggests that self-control is a major contributor to this stark lack of persistence. This study presents the results of a large-scale field experiment (N = 18,043) that examines the effects of a self-directed scheduling nudge designed to promote student persistence in a massive open online course. We find that random assignment to treatment had no effects on near-term engagement and weakly significant negative effects on longer-term course engagement, persistence, and performance. Interestingly, these negative effects are highly concentrated in two groups of students: those who registered close to the first day of class and those with .edu e-mail addresses. We consider several explanations for these findings and conclude that theoretically motivated interventions may interact with the diverse motivations of individual students in possibly unintended ways.

[1]  Kenneth A. Graetz,et al.  Procrastination in Online Courses: Performance and Attitudinal Differences , 2003 .

[2]  Robert F. Boruch,et al.  Moving Through MOOCs , 2014 .

[3]  Mar Pérez-Sanagustín,et al.  Recommending Self-Regulated Learning Strategies Does Not Improve Performance in a MOOC , 2016, L@S.

[4]  Devayani Tirthali,et al.  MOOCs: Expectations and Reality. Full Report. , 2014 .

[5]  R. L. Dipboye,et al.  College students' time management : correlations with academic performance and stress , 1990 .

[6]  Jing Li,et al.  Heterogeneous Paths Through College: Detailed Patterns and Relationships with Graduation and Earnings , 2014 .

[7]  R. Thaler.  Some empirical evidence on dynamic inconsistency , 1981 .

[8]  David I. Laibson,et al.  Golden Eggs and Hyperbolic Discounting , 1997 .

[9]  G. Loewenstein,et al.  Time Discounting and Time Preference: A Critical Review , 2002 .

[10]  Andrew J. Saltarelli,et al.  Who Takes MOOCs , 2016 .

[11]  Richard W. Patterson.  Can behavioral tools improve online student outcomes? Experimental evidence from a massive open online course , 2018, Journal of Economic Behavior & Organization.

[12]  Alan R. Roper.  How Students Develop Online Learning Skills. , 2007 .

[13]  Alexander C. McCormick.  Swirling and Double-Dipping: New Patterns of Student Attendance and Their Implications for Higher Education. , 2003 .

[14]  Justin Reich,et al.  HarvardX and MITx: The First Year of Open Online Courses, Fall 2012-Summer 2013 , 2014 .

[15]  George Veletsianos,et al.  A Systematic Analysis and Synthesis of the Empirical MOOC Literature Published in 2013–2015 , 2016 .

[16]  S. Mullainathan,et al.  Self-Control at Work , 2015, Journal of Political Economy.

[17]  Brent J. Evans,et al.  MOOCs and Persistence: Definitions and Predictors. , 2016 .

[18]  Alberto Bisin,et al.  Present-Bias, Procrastination and Deadlines in a Field Experiment , 2014, Games Econ. Behav..

[19]  Olivier Le Bohec,et al.  Procrastination, participation, and performance in online learning environments , 2011, Comput. Educ..

[20]  Gayle S. Christensen,et al.  The MOOC Phenomenon: Who Takes Massive Open Online Courses and Why? , 2013 .

[21]  Antoine Doucet,et al.  Building engagement for MOOC students: introducing support for time management on online learning platforms , 2014, WWW.

[22]  Gary Charness,et al.  Journal of Economic Behavior & Organization Field and Online Experiments on Self-control , 2022 .

[23]  Dean Karlan,et al.  Commitment Devices , 2009 .

[24]  John Dunlosky,et al.  Study strategies of college students: Are self-testing and scheduling related to achievement? , 2011, Psychonomic Bulletin & Review.

[25]  Gautam Biswas,et al.  Behavior Prediction in MOOCs using Higher Granularity Temporal Information , 2015, L@S.

[26]  J. Angrist,et al.  Identification and Estimation of Local Average Treatment Effects , 1995 .

[27]  Rachel B. Baker,et al.  Persistence Patterns in Massive Open Online Courses (MOOCs) , 2015 .

[28]  B. Bernheim,et al.  Do Consumers Exploit Precommitment Opportunities? Evidence from Natural Experiments Involving Liquor Consumption , 2016 .

[29]  Sherif A. Halawa,et al.  Dropout Prediction in MOOCs using Learner Activity Features , 2014 .

[30]  Peter H. M. P. Roelofsma,et al.  Anomalies in intertemporal choice , 1996 .

[31]  Susan M. Dynarski,et al.  Inequality in postsecondary education , 2011 .

[32]  Marianne M. van den Hurk,et al.  The relation between self-regulated strategies and individual study time, prepared participation and achievement in a problem-based curriculum , 2006 .

[33]  Esther Duflo,et al.  Dis)organization and Success in an Economics MOOC , 2014 .

[34]  D. Ariely,et al.  Procrastination, Deadlines, and Performance: Self-Control by Precommitment , 2002, Psychological science.

[35]  B. K. Britton,et al.  Effects of Time-Management Practices on College Grades , 1991 .

[36]  Dean S. Karlan,et al.  Tying Odysseus to the Mast: Evidence from a Commitment Savings Product in the Philippines , 2005 .

[37]  Jonathan Zinman,et al.  Put Your Money Where Your Butt is: A Commitment Contract for Smoking Cessation , 2009 .

[38]  Janette R. Hill,et al.  Improving online learning: Student perceptions of useful and challenging characteristics , 2004, Internet High. Educ..

[39]  Fiona M. Hollands and Devayani Tirthali.  MOOCs: Expectations and Reality , 2014 .

[40]  Albert Sangrà,et al.  MOOC Design Principles. A Pedagogical Approach from the Learner"s Perspective , 2013 .

[41]  G. Loewenstein,et al.  Anomalies in Intertemporal Choice: Evidence and an Interpretation , 1992 .

[42]  Janette R. Hill,et al.  Overcoming obstacles and creating connections: Community building in Web-based learning environments , 2002, J. Comput. High. Educ..