Sentiment Analysis on Conversations in Collaborative Active Learning as an Early Predictor of Performance

This full research paper studies affective states in students’ verbal conversations in an introductory Computer Science class (CS1) as they work in teams and discuss course content. Research on the cognitive process suggests that social constructs are an essential part of the learning process [1]. This highlights the importance of teamwork in engineering education. Besides cognitive and social constructs, performance evaluation methods are key components of successful team experience. However, measuring students’ individual performance in low-stake teams is a challenge since the main goal of these teams is social construction of knowledge rather than final artifact production. On the other hand, in low-stake teams the small contribution of teamwork to students’ grade might cause students not to collaborate as expected. We study affective metrics of sentiment and subjectivity in collaborative conversations in low-stake teams to identify the correlation between students’ affective states and their performance in CS1 course. The novelty of this research is its focus on students’ verbal conversations in class and how to identify and operationalize affect as a metric that is related to individual performance. We record students’ conversation during low-stake teamwork in multiple sessions throughout the semester. By applying Natural Language Processing (NLP) algorithms, sentiment classes and subjectivity scores are extracted from their speech. The result of this study shows a positive correlation between students’ performance and their positive sentiment as well as the level of subjectivity in speech. The outcome of this research has the potential to serve as a performance predictor in earlier stages of the semester to provide timely feedback to students and enables instructors to make interventions that can lead to student success.

[1]  Fatos Xhafa,et al.  Towards an Emotion Labeling Model to Detect Emotions in Educational Discourse , 2014, 2014 Eighth International Conference on Complex, Intelligent and Software Intensive Systems.

[2]  E. Salas,et al.  The Assessment of Team Performance: Observations and Needs , 2017 .

[3]  Marleen B. Schippers,et al.  Mapping the information flow from one brain to another during gestural communication , 2010, Proceedings of the National Academy of Sciences.

[4]  Celine Latulipe,et al.  Structuring Flipped Classes with Lightweight Teams and Gamification , 2015, SIGCSE.

[5]  Eva Hudlicka,et al.  To feel or not to feel: The role of affect in human-computer interaction , 2003, Int. J. Hum. Comput. Stud..

[6]  Nasrin Dehbozorgi,et al.  Active Learning Design Patterns for CS Education , 2017, ICER.

[7]  Mary Lou Maher,et al.  A Comparison of Lecture-based and Active Learning Design Patterns in CS Education , 2018, 2018 IEEE Frontiers in Education Conference (FIE).

[8]  Winston Bennett,et al.  Performance Measurement : Current Perspectives and Future Challenges , 2006 .

[9]  S. Kozlowski,et al.  Work Groups and Teams in Organizations , 2003 .

[10]  Rhiannon Howe.  A tough nut to crack. , 2015, Lancet psychiatry.

[11]  M. Mukaka,et al.  Statistics corner: A guide to appropriate use of correlation coefficient in medical research. , 2012, Malawi medical journal : the journal of Medical Association of Malawi.

[12]  P. Kutnick,et al.  Emotional and Behavioural Difficulties in Schools: Consideration of Relationships Between Theory and Practice , 1999 .

[13]  Nickolas J. G. Falkner,et al.  Using Learning Analytics to Visualise Computer Science Teamwork , 2015, ITiCSE.

[14]  Anne C. Frenzel,et al.  Measuring emotions in students learning and performance: The Achievement Emotions Questionnaire (AE , 2011 .

[15]  Jakob Heinzle,et al.  Flow of affective information between communicating brains , 2011, NeuroImage.

[16]  D. Perkins,et al.  Individual and Social Aspects of Learning , 1998 .

[17]  Eric Gilbert,et al.  VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text , 2014, ICWSM.

[18]  Jake M. Laguador.  DEVELOPING STUDENTS' ATTITUDE LEADING TOWARDS A LIFE-CHANGING CAREER , 2013 .

[19]  Erkki Sutinen,et al.  Exploiting sentiment analysis to track emotions in students' learning diaries , 2013, Koli Calling '13.

[20]  Anindya Iqbal,et al.  SentiCR: A customized sentiment analysis tool for code review interactions , 2017, 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[21]  Shahaboddin Shamshirband,et al.  Machine Learning-Based Sentiment Analysis for Twitter Accounts , 2018 .

[22]  Douglas B. McLeod,et al.  Research on affect in mathematics education: A reconceptualization. , 1992 .

[23]  Jan Hauke,et al.  Comparison of Values of Pearson's and Spearman's Correlation Coefficients on the Same Sets of Data , 2011 .

[24]  Reinhard Pekrun,et al.  Emotion in Education , 2007 .

[25]  Sônia Maria Guedes-Gondim,et al.  Performance and affects in group problem-solving , 2016 .

[26]  Fatos Xhafa,et al.  Analyzing How Emotion Awareness Influences Students' Motivation, Engagement, Self-Regulation and Learning Outcome , 2016, J. Educ. Technol. Soc..

[27]  James E. Driskell,et al.  Collective Orientation and Team Performance: Development of an Individual Differences Measure , 2010, Hum. Factors.

[28]  Patrick K. Kuiper,et al.  Improving Student Surveys with Natural Language Processing , 2018, 2018 Second IEEE International Conference on Robotic Computing (IRC).

[29]  Dustin K. Jundt,et al.  Teams in organizations: from input-process-output models to IMOI models. , 2005, Annual review of psychology.