How do we model learning at scale ?

Despite a surge of empirical work on student participation in online learning environments, the causal links between the learning-related factors and processes with the desired learning outcomes remain unexplored. This study presents a systematic literature review of approaches to model learning in Massive Open Online Courses offering an analysis of learning related constructs used in the prediction and measurement of student engagement and learning outcome. Based on our literature review, we identify current gaps in the research, including a lack of solid frameworks to explain learning in open online setting. Finally, we put forward a novel framework suitable for open online contexts based on a well-established model of student engagement. Our model is intended to guide future work studying the association between contextual factors (i.e., demographic, classroom, and individual needs), student engagement (i.e., academic, behavioral, cognitive, and affective engagement metrics) and learning outcomes (i.e., academic, social, and affective). The proposed model affords further inter-study comparisons as well as comparative studies with more traditional education models.

[1]  M. Kane The Argument-Based Approach to Validation , 1990 .

[2]  Philip H. Winne,et al.  Studying as self-regulated learning. , 1998 .

[3]  M. Eraut Non-formal learning and tacit knowledge in professional work. , 2000, The British journal of educational psychology.

[4]  P. Goodyear Psychological foundations for networked learning , 2001 .

[5]  George Siemens Connectivism: A Learning Theory for the Digital Age , 2004 .

[6]  Stephen J Ceci,et al.  The rhetoric and reality of gap closing: when the "have-nots" gain but the "haves" gain even more . , 2005, The American psychologist.

[7]  Pamela A. Moss,et al.  Chapter 4: Validity in Educational Assessment , 2006 .

[8]  Knud Illeris,et al.  How We Learn: Learning and Non-Learning in School and Beyond , 2007 .

[9]  R. Azevedo,et al.  A Theoretical Review of Winne and Hadwin’s Model of Self-Regulated Learning: New Perspectives and Directions , 2007 .

[10]  Chris Jones Networked Learning - A social practice perspective , 2008 .

[11]  Arthur C. Graesser,et al.  Cohesion Relationships in Tutorial Dialogue as Predictors of Affective States , 2009, AIED.

[12]  Michelene T. H. Chi,et al.  Active-Constructive-Interactive: A Conceptual Framework for Differentiating Learning Activities , 2009, Top. Cogn. Sci..

[13]  J. Pennebaker,et al.  The Psychological Meaning of Words: LIWC and Computerized Text Analysis Methods , 2010 .

[14]  Maite Taboada,et al.  Lexicon-Based Methods for Sentiment Analysis , 2011, CL.

[15]  Maggie Hartnett,et al.  Examining motivation in online distance learning environments: Complex, multifaceted and situation-dependent , 2011 .

[16]  S. Christenson,et al.  Jingle, Jangle, 1 and Conceptual Haziness 2 : Evolution and Future Directions of the Engagement Construct , 2012 .

[17]  C. Osvaldo Rodriguez,et al.  MOOCs and the AI-Stanford Like Courses: Two Successful and Distinct Course Formats for Massive Open Online Courses. , 2012 .

[18]  Mark Warschauer,et al.  Predicting MOOC performance with Week 1 Behavior , 2014, EDM.

[19]  Carolyn Penstein Rosé,et al.  Sentiment Analysis in MOOC Discussion Forums: What does it tell us? , 2014, EDM.

[20]  E. Xing,et al.  Towards an Integration of Text and Graph Clustering Methods as a Lens for Studying Social Interaction in MOOCs , 2014 .

[21]  Ulrike Cress,et al.  Explaining authors’ contribution to pivotal artifacts during mass collaboration in the Wikipedia’s knowledge base , 2013, International Journal of Computer-Supported Collaborative Learning.

[22]  Carolyn Penstein Rosé,et al.  Linguistic Reflections of Student Engagement in Massive Open Online Courses , 2014, ICWSM.

[23]  Rebecca Eynon,et al.  Communication patterns in massively open online courses , 2014, Internet High. Educ..

[24]  Conrad S. Tucker,et al.  Mining Student-Generated Textual Data In MOOCS And Quantifying Their Effects on Student Performance and Learning Outcomes , 2014 .

[25]  Michael D. Ekstrand,et al.  Teaching recommender systems at large scale: evaluation and lessons learned from a hybrid MOOC , 2014, L@S.

[26]  Jane Sinclair,et al.  Learning Technology for Education in Cloud. MOOC and Big Data , 2014, Communications in Computer and Information Science.

[27]  Erik Duval,et al.  Success, activity and drop-outs in MOOCs an exploratory study on the UNED COMA courses , 2014, LAK.

[28]  Andrew D. Ho,et al.  Changing “Course” , 2014 .

[29]  Mark Warschauer,et al.  Social Positioning and Performance in MOOCs , 2014, EDM.

[30]  Linda Corrin,et al.  Visualizing patterns of student engagement and performance in MOOCs , 2014, LAK.

[31]  Sherif Halawa,et al.  Attrition and Achievement Gaps in Online Learning , 2015, L@S.

[32]  Linda Corrin,et al.  Predicting success: how learners' prior knowledge, skills and activities predict MOOC performance , 2015, LAK.

[33]  Joseph Jay Williams,et al.  Beyond Prediction: Towards Automatic Intervention in MOOC Student Stop-out , 2015, EDM.

[34]  Danielle S. McNamara,et al.  Language to Completion: Success in an Educational Data Mining Massive Open Online Class , 2015, EDM.

[35]  Gautam Biswas,et al.  Behavior Prediction in MOOCs using Higher Granularity Temporal Information , 2015, L@S.

[36]  Patrick Jermann,et al.  Identifying Styles and Paths toward Success in MOOCs , 2015, EDM.

[37]  Ming Ming Chiu,et al.  Effects of sequences of socially regulated learning on group performance , 2015, LAK.

[38]  Deborah L Engle,et al.  Coursera's Introductory Human Physiology Course: Factors That Characterize Successful Completion of a MOOC. , 2015 .

[39]  Dragan Gasevic,et al.  Importance of Theory in Learning Analytics in Formal and Workplace Settings , 2015 .

[40]  A. Wise,et al.  Why Theory Matters More than Ever in the Age of Big Data , 2015, J. Learn. Anal..

[41]  Kalyan Veeramachaneni,et al.  Transfer Learning for Predictive Models in Massive Open Online Courses , 2015, AIED.

[42]  Patrick Jermann,et al.  Conscientious Behaviour, Flexibility and Learning in Massive Open On-Line Courses , 2015 .

[43]  J. Greene,et al.  Predictors of Retention and Achievement in a Massive Open Online Course , 2015 .

[44]  Garry Robins,et al.  Relational event models for social learning in MOOCs , 2015, Soc. Networks.

[45]  Ryan S. Baker,et al.  Content or platform: Why do students complete MOOCs? , 2015 .

[46]  Patrick Jermann,et al.  MOOC Video Interaction Patterns: What Do They Tell Us? , 2015, EC-TEL.

[47]  N. Selwyn,et al.  Massive Open Online Change? Exploring the Discursive Construction of the "MOOC" in Newspapers. , 2015 .

[48]  Katy Jordan,et al.  Massive Open Online Course Completion Rates Revisited: Assessment, Length and Attrition , 2015 .

[49]  J. Vickers,et al.  Relationship between participants’ level of education and engagement in their completion of the Understanding Dementia Massive Open Online Course , 2015, BMC medical education.

[50]  Kenneth R. Koedinger,et al.  Learning is Not a Spectator Sport: Doing is Better than Watching for Learning from a MOOC , 2015, L@S.

[51]  George Siemens,et al.  Preparing for the Digital University: A Review of the History and Current State of Distance, Blended and Online Learning , 2015 .

[52]  Tawanna Dillahunt,et al.  Learn With Friends: The Effects of Student Face-to-Face Collaborations on Massive Open Online Course Activities , 2015, L@S.

[53]  Justine Cassell,et al.  Connecting the Dots: Predicting Student Grade Sequences from Bursty MOOC Interactions over Time , 2015, L@S.

[54]  René F. Kizilcec,et al.  Motivation as a Lens to Understand Online Learners , 2015, ACM Trans. Comput. Hum. Interact..

[55]  Carolyn Penstein Rosé,et al.  Investigating How Student's Cognitive Behavior in MOOC Discussion Forum Affect Learning Gains , 2015, EDM.

[56]  Donatella Persico,et al.  Methodological approaches in MOOC research: Retracing the myth of Proteus , 2015, Br. J. Educ. Technol..

[57]  Carolyn Penstein Rosé,et al.  Exploring the Effect of Confusion in Discussion Forums of Massive Open Online Courses , 2015, L@S.

[58]  Dragan Gasevic,et al.  Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success , 2016, Internet High. Educ..

[59]  L. Czerniewicz,et al.  Learning through engagement: MOOCs as an emergent form of provision , 2016 .

[60]  Justin Reich,et al.  The Civic Mission of MOOCs: Measuring Engagement across Political Differences in Forums , 2016, L@S.

[61]  Angela L. Duckworth,et al.  Advanced, Analytic, Automated (AAA) Measurement of Engagement During Learning , 2017, Educational psychologist.

[62]  Gregory Mahr Validation , 2019, Academic Psychiatry.