An Identification of Learners' Confusion through Language and Discourse Analysis

The substantial growth of online learning, in particular, Massively Open Online Courses (MOOCs), supports research into the development of better models for effective learning. Learner 'confusion' is among one of the identified aspects which impacts the overall learning process, and ultimately, course attrition. Confusion for a learner is an individual state of bewilderment and uncertainty of how to move forward. The majority of recent works neglect the 'individual' factor and measure the influence of community-related aspects (e.g. votes, views) for confusion classification. While this is a useful measure, as the popularity of one's post can indicate that many other students have similar confusion regarding course topics, these models neglect the personalised context, such as individual's affect or emotions. Certain physiological aspects (e.g. facial expressions, heart rate) have been utilised to classify confusion in small to medium classrooms. However, these techniques are challenging to adopt to MOOCs. To bridge this gap, we propose an approach solely based on language and discourse aspects of learners, which outperforms the previous models. We contribute through the development of a novel linguistic feature set that is predictive for confusion classification. We train the confusion classifier using one domain, successfully applying it across other domains.

[1]  Aditya Johri,et al.  Needle in a haystack: Identifying learner posts that require urgent response in MOOC discussion forums , 2018, Comput. Educ..

[2]  Carolyn Penstein Rosé,et al.  Exploring the Effect of Student Confusion in Massive Open Online Courses , 2016, EDM.

[3]  A. Graesser,et al.  Dynamics of affective states during complex learning , 2012 .

[4]  Jihie Kim,et al.  Predicting Learner's Project Performance with Dialogue Features in Online Q&A Discussions , 2012, ITS.

[5]  Dan Goldwasser,et al.  Predicting Instructor’s Intervention in MOOC forums , 2014, ACL.

[6]  Danielle S McNamara,et al.  Sentiment Analysis and Social Cognition Engine (SEANCE): An automatic tool for sentiment, social cognition, and social-order analysis , 2017, Behavior research methods.

[7]  Collin Lynch,et al.  Your Actions or Your Associates? Predicting Certification and Dropout in MOOCs with Behavioral and Social Features , 2018, EDM.

[8]  Arthur C Graesser,et al.  Learning, thinking, and emoting with discourse technologies. , 2011, The American psychologist.

[9]  M. C. Templin Certain language skills in children : their development and interrelationships , 1957 .

[10]  Philip M. McCarthy,et al.  MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment , 2010, Behavior research methods.

[11]  Kristy Elizabeth Boyer,et al.  Automatically Recognizing Facial Expression: Predicting Engagement and Frustration , 2013, EDM.

[12]  Carlos Delgado Kloos,et al.  Prediction in MOOCs: A Review and Future Research Directions , 2019, IEEE Transactions on Learning Technologies.

[13]  James W. Pennebaker,et al.  Linguistic Inquiry and Word Count (LIWC2007) , 2007 .

[14]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[15]  A. Graesser,et al.  Confusion can be beneficial for learning. , 2014 .

[16]  Arthur C. Graesser,et al.  Who Benefits from Confusion Induction during Learning? An Individual Differences Cluster Analysis , 2013, AIED.

[17]  Nickolas J. G. Falkner,et al.  A comprehensive text analysis of lecture slides to generate concept maps , 2017, Comput. Educ..

[18]  Carolyn Penstein Rosé,et al.  Exploring the Effect of Confusion in Discussion Forums of Massive Open Online Courses , 2015, L@S.

[19]  Min-Yen Kan,et al.  Learning Instructor Intervention from MOOC Forums: Early Results and Issues , 2015, EDM.

[20]  Sidney D'Mello,et al.  Confusion and its dynamics during device comprehension with breakdown scenarios. , 2014, Acta psychologica.

[21]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[22]  Noureddine Elouazizi Point-of-View Mining and Cognitive Presence in MOOCs: A (Computational) Linguistics Perspective , 2014, EMNLP 2014.

[23]  Andreas Paepcke,et al.  YouEDU: Addressing Confusion in MOOC Discussion Forums by Recommending Instructional Video Clips , 2015, EDM.

[24]  Arthur C. Graesser,et al.  Automated Evaluation of Text and Discourse with Coh-Metrix: Introduction , 2014 .

[25]  Arthur C. Graesser,et al.  Automatic detection of learner’s affect from conversational cues , 2008, User Modeling and User-Adapted Interaction.

[26]  Aneesha Bakharia,et al.  Towards Cross-domain MOOC Forum Post Classification , 2016, L@S.

[27]  Lori Lockyer,et al.  Understanding Difficulties and Resulting Confusion in Learning: An Integrative Review , 2018, Front. Educ..

[28]  Vincent Aleven,et al.  Towards Sensor-Free Affect Detection in Cognitive Tutor Algebra. , 2012, EDM 2012.

[29]  Eric Gilbert,et al.  VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text , 2014, ICWSM.

[30]  Lori Lockyer,et al.  Inside Out , 2017 .

[31]  Carolyn Penstein Rosé,et al.  Towards triggering higher-order thinking behaviors in MOOCs , 2016, LAK.

[32]  Alyssa Friend Wise,et al.  Identifying Content-Related Threads in MOOC Discussion Forums , 2015, L@S.

[33]  Minjuan Wang,et al.  Affective e-Learning: Using "Emotional" Data to Improve Learning in Pervasive Learning Environment , 2009, J. Educ. Technol. Soc..

[34]  Snigdha Chaturvedi,et al.  Learner Affect Through the Looking Glass: Characterization and Detection of Confusion in Online Courses , 2017, EDM.

[35]  Guido Zuccon,et al.  Classifying help seeking behaviour in online communities , 2017, LAK.

[36]  Carolyn Penstein Rosé,et al.  Sentiment Analysis in MOOC Discussion Forums: What does it tell us? , 2014, EDM.

[37]  Gráinne Conole,et al.  MOOCs as disruptive technologies: strategies for enhancing the learner experience and quality of MOOCs Los MOOC como tecnologías disruptivas: estrategias para mejorar la experiencia de aprendizaje y la calidad de los MOOC. , 2016 .

[38]  Hinrich Schütze,et al.  Introduction to information retrieval , 2008 .

[39]  Marold Wosnitza,et al.  Origin, direction and impact of emotions in social online learning , 2005 .

[40]  Laura K. Allen,et al.  Analyzing Discourse Processing Using a Simple Natural Language Processing Tool , 2014 .

[41]  Saif Mohammad,et al.  CROWDSOURCING A WORD–EMOTION ASSOCIATION LEXICON , 2013, Comput. Intell..

[42]  Arthur C. Graesser,et al.  Language and Discourse Are Powerful Signals of Student Emotions during Tutoring , 2012, IEEE Transactions on Learning Technologies.

[43]  Linda Corrin,et al.  Using learning analytics to explore help-seeking learner profiles in MOOCs , 2017, LAK.