A Learning Analytics Methodology for Understanding Social Interactions in MOOCs

One of the characteristics of massive open online courses (MOOCs) is that the overall number of social interactions tend to be higher than in traditional courses, hindering the analysis of social learning. Learners typically ask or answer questions using the forum. This makes messages a rich source of information, which can be used to infer learners’ behavior and outcomes. It is not feasible for teachers to process all forum messages and automated tools and analysis are required. Although there are some tools for analyzing learners’ interactions, there is a need for methodologies and integrated tools that help to interpret the learning process based on social interactions in the forum. This paper presents the 3S (Social, Sentiments, Skills) learning analytics methodology for analyzing forum interactions in MOOCs. This methodology considers a temporal analysis combining the social, sentiments, and skill dimensions that can be extracted from forum data. We also introduce LATƎS, a learning analytics tool for edX/Open edX related to the three dimensions (3S), which includes visualizations to guide the proposed methodology. We apply the 3S methodology and the tool to an MOOC on Java programming. Results showed, among others, the action–reaction effect produced when learners increase their participation after instructor's events. Moreover, a decrease of positive sentiments over time and before deadlines of open-ended assignments was also observed and that there were certain skills, which caused more troubles (e.g., arrays and loops). These results acknowledge the importance of using a learning analytics methodology to detect problems in MOOCs.

[1]  Geert-Jan Houben,et al.  On the Impact of Personality in Massive Open Online Learning , 2016, UMAP.

[2]  María Jesús Rodríguez-Triana,et al.  Perceiving Learning at a Glance: A Systematic Literature Review of Learning Dashboard Research , 2017, IEEE Transactions on Learning Technologies.

[3]  Carlos Delgado Kloos,et al.  Delving into Participants’ Profiles and Use of Social Tools in MOOCs , 2014, IEEE Transactions on Learning Technologies.

[4]  Rebecca Ferguson,et al.  Social Learning Analytics , 2012, J. Educ. Technol. Soc..

[5]  José A. Ruipérez-Valiente,et al.  Scaling to Massiveness With ANALYSE: A Learning Analytics Tool for Open edX , 2017, IEEE Transactions on Human-Machine Systems.

[6]  Marek Hatala,et al.  A qualitative evaluation of evolution of a learning analytics tool , 2012, Comput. Educ..

[7]  Stephen Travis Pope,et al.  A Description of the Model-View-Controller User Interface Paradigm in the Smalltalk-80 System , 1998 .

[8]  Albrecht Fortenbacher,et al.  Predicting students' success based on forum activities in MOOCs , 2015, 2015 IEEE 8th International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS).

[9]  Carlos Delgado Kloos,et al.  FLINN: A Framework to Characterize Technology Enhanced Formal, Non-Formal and Informal Learning Situations , 2014, Int. J. Hum. Cap. Inf. Technol. Prof..

[10]  Roy Williams,et al.  Blogs and forums as communication and learning tools in a MOOC , 2010 .

[11]  Jane Sinclair,et al.  Exploring the use of MOOC discussion forums , 2014 .

[12]  Jeff Grann,et al.  Competency map: visualizing student learning to promote student success , 2014, LAK.

[13]  Stephanie Boehm,et al.  Information Dashboard Design The Effective Visual Communication Of Data , 2016 .

[14]  Carlos Delgado Kloos,et al.  Sentiment analysis in MOOCs: A case study , 2018, 2018 IEEE Global Engineering Education Conference (EDUCON).

[15]  Daniel Bodemer,et al.  Selective reading of large online forum discussions: The impact of rating visualizations on navigation and learning , 2015, Comput. Hum. Behav..

[16]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[17]  Lorenzo Vigentini,et al.  Toward the development of a dynamic dashboard for FutureLearn MOOCs , 2016, ASCILITE Publications.

[18]  Carolyn Penstein Rosé,et al.  Investigating How Student's Cognitive Behavior in MOOC Discussion Forum Affect Learning Gains , 2015, EDM.

[19]  Lenh Hung Son Do Supporting Reflection and Classroom Orchestration with Tangible Tabletops , 2012 .

[20]  Carlos Delgado Kloos,et al.  Analysing the Impact of Built-In and External Social Tools in a MOOC on Educational Technologies , 2013, EC-TEL.

[21]  Dwen-Ren Tsai,et al.  Social interaction feedback system for the smart classroom , 2012, 2012 IEEE International Conference on Consumer Electronics (ICCE).

[22]  Alejandro Peña-Ayala,et al.  Learning Analytics: Fundaments, Applications, and Trends , 2017 .

[23]  Dan Goldwasser,et al.  Predicting Instructor’s Intervention in MOOC forums , 2014, ACL.

[24]  Giuseppe Carenini,et al.  ConVis: A Visual Text Analytic System for Exploring Blog Conversations , 2014, Comput. Graph. Forum.

[25]  Bing Liu,et al.  Mining and summarizing customer reviews , 2004, KDD.

[26]  Carlos Delgado Kloos,et al.  ALAS-KA: A learning analytics extension for better understanding the learning process in the Khan Academy platform , 2015, Comput. Hum. Behav..

[27]  James R. Foulds,et al.  Weakly Supervised Models of Aspect-Sentiment for Online Course Discussion Forums , 2015, ACL.

[28]  Qing Chen,et al.  VisMOOC: Visualizing video clickstream data from massive open online courses , 2015, 2014 IEEE Conference on Visual Analytics Science and Technology (VAST).

[29]  Tzone-I Wang,et al.  Java learning object ontology , 2005, Fifth IEEE International Conference on Advanced Learning Technologies (ICALT'05).

[30]  Manolis Mavrikis,et al.  The impact of feedback on students' affective states , 2015, EDM.

[31]  Achilles Kameas,et al.  Modeling the Knowledge Domain of the Java Programming Language as an Ontology , 2012, ICWL.

[32]  Aneesha Bakharia,et al.  Towards Cross-domain MOOC Forum Post Classification , 2016, L@S.

[33]  Erik Duval,et al.  Learning Analytics Dashboard Applications , 2013 .

[34]  Qing Chen,et al.  Visual Analytics for MOOC Data , 2015, IEEE Computer Graphics and Applications.

[35]  Christian Gütl,et al.  Attrition in MOOC: Lessons Learned from Drop-Out Students , 2014, LTEC@KMO.

[36]  Erik Duval,et al.  Learning dashboards: an overview and future research opportunities , 2013, Personal and Ubiquitous Computing.

[37]  Carlos Delgado Kloos,et al.  Analysing the predictive power for anticipating assignment grades in a massive open online course , 2018, Behav. Inf. Technol..

[38]  Carolyn Penstein Rosé,et al.  Exploring the Effect of Student Confusion in Massive Open Online Courses , 2016, EDM.