Crowd-sourced learning in MOOCs: learning analytics meets measurement theory

This paper illustrated the promise of the combination of measurement theory and learning analytics for understanding effective MOOC learning. It reports findings from a study of whether and how MOOC log file data can assist in understanding how MOOC participants use (often) messy, chaotic forums to support complex, unpredictable, contingent learning processes. It is argued that descriptions of posting, voting and viewing behaviours do not in and of themselves provide insights about how learning is generated in MOOC forums. Rather, it is hypothesised that there is a skill involved in using forums to learn; that theory-informed descriptions of this skill illustrate how MOOC participants use forums differently as they progress from novice to expert; that the skill progression can be validated through the use of forum log file data; and that log file data can also be used to assess an individual MOOC participant's position in relation to this progression -- that is, to measure an individual's skill in learning through forums and similar educational settings. These hypotheses were examined using data drawn from forums in a large MOOC run at the University of Melbourne in 2013.

[1]  Helene Fournier,et al.  A pedagogy of abundance or a pedagogy to support human beings? Participant support on massive open online courses , 2011 .

[2]  S. Dreyfus,et al.  A Five-Stage Model of the Mental Activities Involved in Directed Skill Acquisition , 1980 .

[3]  Ronald K. Hambleton.  Item response theory: a broad psychometric framework for measurement advances 1, 2 , 1994 .

[4]  Nabeel Gillani,et al.  Learner Communications in Massively Open Online Courses , 2013 .

[5]  Nobuko Fujita,et al.  at UWindsor , 2018 .

[6]  Carolyn Penstein Rosé,et al.  Sentiment Analysis in MOOC Discussion Forums: What does it tell us? , 2014, EDM.

[7]  Margaret Wu,et al.  ACER conquest: generalised item response modelling software , 1998 .

[8]  D. Andrich.  A rating formulation for ordered response categories , 1978 .

[9]  G. Masters.  A rasch model for partial credit scoring , 1982 .

[10]  Erling B. Andersen,et al.  Sufficient statistics and latent trait models , 1977 .

[11]  Patrick Griffin,et al.  Assessment and Teaching of 21st Century Skills , 2011 .

[12]  Hélène Fournier,et al.  The value of learning analytics to networked learning on a personal learning environment , 2011, LAK.

[13]  Sarah A. Webster,et al.  Learner Participation and Engagement in Open Online Courses: Insights from the Peer 2 Peer University , 2013 .

[14]  George Siemens.  Connectivism: A Learning Theory for the Digital Age , 2004 .

[15]  Alexandra Pickett,et al.  Online learner self-regulation: Learning presence viewed through quantitative content- and social network analysis , 2013 .

[16]  Patrick Griffin,et al.  Assessment and teaching of 21st century skills : methods and approach , 2015 .

[17]  Marion Waite,et al.  Liminal Participants and Skilled Orienteers: Learner Participation in a MOOC for New Lecturers , 2013 .

[18]  Bonnie Stewart,et al.  Massiveness + Openness = New Literacies of Participation? , 2013 .

[19]  Zhenming Liu,et al.  Learning about Social Learning in MOOCs: From Statistical Analysis to Generative Model , 2013, IEEE Transactions on Learning Technologies.

[20]  B. Wright.  Reasonable mean-square fit values , 1994 .

[21]  Rita Kop,et al.  The Challenges to Connectivist Learning on Open Online Networks: Learning Experiences during a Massive Open Online Course , 2011 .

[22]  Allison Littlejohn,et al.  Merlot Journal of Online Learning and Teaching Patterns of Engagement in Connectivist Moocs , 2022 .

[23]  Rebecca Eynon,et al.  Structural limitations of learning in a crowd: communication vulnerability and information diffusion in MOOCs , 2014, Scientific Reports.