Detecting Collaborative Learning Through Emotions: An Investigation Using Facial Expression Recognition

Providing adaptive feedback to learners engaging in collaborative learning activities is one research topic in the development of intelligent tutoring systems. However, there is a need to investigate how to systematically evaluate a learner’s activities and provide feedback on them. The present study investigates how emotional states, detected through facial recognition, can be utilized to capture the learning process in a simple jigsaw-type collaborative task. It was predicted that when learners argue with each other and reason deeply, they may experience several emotional states such as positive and negative states. The results show that when learners work harder on developing a mutual understanding through conflictive interaction, negative emotions can be used to predict this process. This study contributes to the knowledge of how emotional states detected by facial recognition technology can be applied to predict learning process in conflictive tasks. Moreover, these empirical results will impact the development of adaptive feedback mechanisms for intelligent tutoring systems for collaborative learning.

[1]  Danielle S. McNamara,et al.  iSTART: A Web-based tutor that teaches self-explanation and metacognitive reading strategies. , 2007 .

[2]  L. Vygotsky.  Mind in Society: The Development of Higher Psychological Processes: Harvard University Press , 1978 .

[3]  R. Azevedo,et al.  Does training on self-regulated learning facilitate students' learning with hypermedia? , 2004 .

[4]  N. Miyake,et al.  Cognitively active externalization for situated reflection , 2002 .

[5]  Marco Wiering,et al.  A Model Based Method for Automatic Facial Expression Recognition , 2005, ECML.

[6]  Etienne Wenger,et al.  Situated Learning: Legitimate Peripheral Participation , 1991 .

[7]  M. Chi,et al.  Eliciting Self‐Explanations Improves Understanding , 1994 .

[8]  Nikol Rummel,et al.  A rating scheme for assessing the quality of computer-supported collaboration processes , 2007, Int. J. Comput. Support. Collab. Learn..

[9]  J. Metcalfe,et al.  Intuition in insight and noninsight problem solving , 1987, Memory & cognition.

[10]  Yugo Hayashi,et al.  Gaze Feedback and Pedagogical Suggestions in Collaborative Learning - Investigation of Explanation Performance on Self's Concept in a Knowledge Integration Task , 2018, ITS.

[11]  Arthur C. Graesser,et al.  When Are Tutorial Dialogues More Effective Than Reading? , 2007, Cogn. Sci..

[12]  Arthur C. Graesser,et al.  Computational Analyses of Multilevel Discourse Comprehension , 2011, Top. Cogn. Sci..

[13]  Arthur C. Graesser,et al.  AutoTutor: an intelligent tutoring system with mixed-initiative dialogue , 2005, IEEE Transactions on Education.

[14]  Gautam Biswas,et al.  Designing Learning by Teaching Agents: The Betty's Brain System , 2008, Int. J. Artif. Intell. Educ..

[15]  Naomi Miyake.  Constructive interaction and the iterative process of understanding , 1986 .

[16]  Vincent Aleven,et al.  Intelligent Tutoring Goes To School in the Big City , 1997 .

[17]  Stephen M. Fiore,et al.  Toward an Understanding of Macrocognition in Teams: Predicting Processes in Complex Collaborative Contexts , 2010, Hum. Factors.

[18]  Lei Liu,et al.  Interdisciplinary research agenda in support of assessment of collaborative problem solving: lessons learned from developing a Collaborative Science Assessment Prototype , 2017, Comput. Hum. Behav..

[19]  Evelyn Yarzebinski,et al.  Studying the Effect of a Competitive Game Show in a Learning by Teaching Environment , 2013, International Journal of Artificial Intelligence in Education.

[20]  Yugo Hayashi.  Coordinating Knowledge Integration with Pedagogical Agents - Effects of Agent Gaze Gestures and Dyad Synchronization , 2016, ITS.

[21]  A. Graesser,et al.  Confusion can be beneficial for learning. , 2014 .

[22]  Yugo Hayashi.  On Pedagogical Effects of Learner-Support Agents in Collaborative Interaction , 2012, ITS.

[23]  Gautam Biswas,et al.  LEARNING BY TEACHING: A NEW AGENT PARADIGM FOR EDUCATIONAL SOFTWARE , 2005, Appl. Artif. Intell..

[24]  Friedrich W. Hesse,et al.  A Framework for Teachable Collaborative Problem Solving Skills , 2015 .

[25]  Yugo Hayashi.  Togetherness: Multiple Pedagogical Conversational Agents as Companions in Collaborative Learning , 2014, Intelligent Tutoring Systems.

[26]  Yugo Hayashi,et al.  The Power of a "Maverick" in Collaborative Problem Solving: An Experimental Investigation of Individual Perspective-Taking Within a Group. , 2018, Cognitive science.

[27]  Carolyn Penstein Rosé,et al.  Architecture for Building Conversational Agents that Support Collaborative Learning , 2011, IEEE Transactions on Learning Technologies.