Deconstructing disengagement: analyzing learner subpopulations in massive open online courses

As MOOCs grow in popularity, the relatively low completion rates of learners has been a central criticism. This focus on completion rates, however, reflects a monolithic view of disengagement that does not allow MOOC designers to target interventions or develop adaptive course features for particular subpopulations of learners. To address this, we present a simple, scalable, and informative classification method that identifies a small number of longitudinal engagement trajectories in MOOCs. Learners are classified based on their patterns of interaction with video lectures and assessments, the primary features of most MOOCs to date. In an analysis of three computer science MOOCs, the classifier consistently identifies four prototypical trajectories of engagement. The most notable of these is the learners who stay engaged through the course without taking assessments. These trajectories are also a useful framework for the comparison of learner engagement between different course structures or instructional approaches. We compare learners in each trajectory and course across demographics, forum participation, video access, and reports of overall experience. These results inform a discussion of future interventions, research, and design directions for MOOCs. Potential improvements to the classification mechanism are also discussed, including the introduction of more fine-grained analytics.

[1]  Henry L. Roediger,et al.  Generalizing test-enhanced learning from the laboratory to the classroom , 2007, Psychonomic bulletin & review.

[2]  B. J. Fogg,et al.  A behavior model for persuasive design , 2009, Persuasive '09.

[3]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[4]  J. Hayden.  Cognitive Surplus: Creativity and Generosity in a Connected Age , 2011 .

[5]  Thomas C. Reeves,et al.  Reframing research on learning with technology: in search of the meaning of cognitive tools , 2007 .

[6]  Sara Goldrick-Rab,et al.  Challenges and Opportunities for Improving Community College Student Success , 2010 .

[7]  S. Spencer,et al.  Stereotype Threat and Women's Math Performance , 1999 .

[8]  Clifford Nass,et al.  Cognitive control in media multitaskers , 2009, Proceedings of the National Academy of Sciences.

[9]  S. Spencer,et al.  Contending with group image: The psychology of stereotype and social identity threat , 2002 .

[10]  P. Black,et al.  Assessment and Classroom Learning , 1998 .

[11]  Raymond G. Hoffmann,et al.  A Handbook of Statistical Analyses Using Stata, 2ndedn,Sophia Rabe-Hesketh and Brian Everitt,Chapman&Hall/CRC,London,U.K.,1998. No. of pages: xi+215. Price:£19.95. ISBN 0-849-30387-7 , 2002 .

[12]  Cristina Conati,et al.  Automatic Recognition of Learner Groups in Exploratory Learning Environments , 2006, Intelligent Tutoring Systems.

[13]  C. Palau,et al.  ASSESSMENT IN EDUCATION , 2015 .

[14]  Peter Brusilovsky,et al.  User Models for Adaptive Hypermedia and Adaptive Educational Systems , 2007, The Adaptive Web.

[15]  Brian Everitt,et al.  A Handbook of Statistical Analyses using SAS SECOND EDITION , 2007 .

[16]  C. Osvaldo Rodriguez,et al.  MOOCs and the AI-Stanford Like Courses: Two Successful and Distinct Course Formats for Massive Open Online Courses. , 2012 .

[17]  P. Rousseeuw.  Silhouettes: a graphical aid to the interpretation and validation of cluster analysis , 1987 .

[18]  J. Church.  Human Development Report , 2001 .

[19]  Daniel D. Suthers,et al.  Computer-supported collaborative learning: An historical perspective , 2006 .

[20]  C. Dweck.  Mindset: The New Psychology of Success , 2006 .

[21]  Tammy Schellens,et al.  Content analysis schemes to analyze transcripts of online asynchronous discussion groups: A review , 2006, Comput. Educ..

[22]  J. Nagaoka,et al.  Teaching Adolescents To Become Learners The Role of Noncognitive Factors in Shaping School Performance: A Critical Literature Review , 2013 .

[23]  Kye-Hyoung Lee,et al.  Community College Student Engagement Patterns , 2011 .

[24]  Ann L. Brown,et al.  How people learn: Brain, mind, experience, and school. , 1999 .

[25]  George Siemens.  Connectivism: A Learning Theory for the Digital Age , 2004 .

[26]  Geoffrey L. Cohen,et al.  Academic Tenacity: Mindsets and Skills that Promote Long-Term Learning. , 2014 .

[27]  P. Resnick,et al.  Building Successful Online Communities: Evidence-Based Social Design , 2012 .

[28]  P. Bahr,et al.  The Bird’s Eye View of Community Colleges: A Behavioral Typology of First-Time Students Based on Cluster Analytic Classification , 2010 .

[29]  Stephen Downes,et al.  New Technology Supporting Informal Learning , 2010 .

[30]  B. Everitt,et al.  A Handbook of Statistical Analyses using R , 2006 .