Multi-attribute Categorization of MOOC Forum Posts and Applications to Conversational Agents

Discussion forums are among the most common interaction tools offered by MOOCs. Nevertheless, due to the high number of students enrolled and the relatively small number of tutors, it is virtually impossible for instructors to effectively monitor and moderate them. For this reason, teacher-guided instructional scaffolding activities may be very limited, even impossible in such environments. On the other hand, students who seek to clarify concepts may not get the attention they need, and lack of responsiveness often favors abandonment. In order to mitigate these issues, we propose in this work a multi-attribute text categorization tool able to automatically detect useful information from MOOC forum posts including intents, topics covered, sentiment polarity, level of confusion and urgency. Extracted information may be used directly by instructors for moderating and planning their interventions as well as input for conversational software agents able to engage learners in guided, constructive discussions through natural language. The results of an experiment aimed at evaluating the performance of the proposed approach on an existing dataset are also presented, as well as the description of an application scenario that exploits the extracted information within a conversation agents’ framework.

[1]  Guy Lapalme,et al.  A systematic analysis of performance measures for classification tasks , 2009, Inf. Process. Manag..

[2]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[3]  Carolyn Penstein Rosé,et al.  Exploring the Effect of Confusion in Discussion Forums of Massive Open Online Courses , 2015, L@S.

[4]  Yannis A. Dimitriadis,et al.  Conversational Agents as Group-Teacher Interaction Mediators in MOOCs , 2018, 2018 Learning With MOOCS (LWMOOCS).

[5]  Shirley Williams,et al.  MOOCs: A systematic study of the published literature 2008-2012 , 2013 .

[6]  Fiona M. Hollands and Devayani Tirthali MOOCs: Expectations and Reality , 2014 .

[7]  Pawel Cichosz,et al.  A Case Study in Text Mining of Discussion Forum Posts: Classification with Bag of Words and Global Vectors , 2018, Int. J. Appl. Math. Comput. Sci..

[8]  Changki Lee,et al.  Fine-Grained Named Entity Recognition Using Conditional Random Fields for Question Answering , 2006, AIRS.

[9]  Carolyn Penstein Rosé,et al.  Architecture for Building Conversational Agents that Support Collaborative Learning , 2011, IEEE Transactions on Learning Technologies.

[10]  Matteo Gaeta,et al.  An Ontology-Based Approach for Context-Aware E-learning , 2011, 2011 Third International Conference on Intelligent Networking and Collaborative Systems.

[11]  Nicola Capuano,et al.  Ontology Extraction from Existing Educational Content to Improve Personalized e-Learning Experiences , 2009, 2009 IEEE International Conference on Semantic Computing.

[12]  David Masip,et al.  Enabling Automatic Just-in-time Evaluation of In-class Discussions in On-line Collaborative Learning Practices , 2009, J. Digit. Inf. Manag..

[13]  Alaa El-Halees,et al.  Mining Opinions in User-Generated Contents to Improve Course Evaluation , 2011, ICSECS.

[14]  Andreas Paepcke,et al.  YouEDU: Addressing Confusion in MOOC Discussion Forums by Recommending Instructional Video Clips , 2015, EDM.

[15]  Carolyn Penstein Rosé,et al.  Sentiment Analysis in MOOC Discussion Forums: What does it tell us? , 2014, EDM.

[16]  Nicola Capuano,et al.  A Fuzzy Group Decision Making Model for Ordinal Peer Assessment , 2017, IEEE Transactions on Learning Technologies.

[17]  Hinrich Schütze,et al.  Introduction to information retrieval , 2008 .

[18]  Haji Binali,et al.  A new significant area: Emotion detection in E-learning using opinion mining techniques , 2009, 2009 3rd IEEE International Conference on Digital Ecosystems and Technologies.

[19]  Quoc V. Le,et al.  Distributed Representations of Sentences and Documents , 2014, ICML.

[20]  Nicola Capuano,et al.  Adaptive Peer Grading and Formative Assessment , 2017 .

[21]  Hongfei Lin,et al.  A Convolution-LSTM-Based Deep Neural Network for Cross-Domain MOOC Forum Post Classification , 2017, Inf..

[22]  Stavros N. Demetriadis,et al.  Conversational Agents Improve Peer Learning through Building on Prior Knowledge , 2017, J. Educ. Technol. Soc..

[23]  David George Glance,et al.  The pedagogical foundations of massive open online courses , 2013, First Monday.

[24]  Fabrizio Sebastiani,et al.  Machine learning in automated text categorization , 2001, CSUR.

[25]  Nicola Capuano,et al.  Towards Adaptive Peer Assessment for MOOCs , 2015, 2015 10th International Conference on P2P, Parallel, Grid, Cloud and Internet Computing (3PGCIC).

[26]  Donna Charlevoix,et al.  Do professors matter?: using an a/b test to evaluate the impact of instructor involvement on MOOC student outcomes , 2014, L@S.