Mining Bodily Patterns of Affective Experience during Learning

We investigated 28 learners’ postural patterns associated with naturally occurring episodes of boredom, flow/engagement, confusion, frustration, and delight during a tutoring session with AutoTutor, a dialoguebased intelligent tutoring system. Training and validation data were collected in a learning session with AutoTutor, after which the learners’ affective states (i.e., emotions) were rated by the learner, a peer, and two trained judges. An automated body pressure measurement system was used to capture the pressure exerted by the learner on the seat and back of a chair during the tutoring session. We extracted 16 posture-related features that focused on the pressure exerted along with the magnitude and direction of changes in pressure during emotional experiences. Binary logistic regression models yielded medium sized effects in discriminating the affective states from neutral. An analysis of the parameters of the models indicated that the affective states were manifested by three unique postural configurations and a general increase in movement (when compared to neutral).

[1]  Kasia Muldner,et al.  Emotion Sensors Go To School , 2009, AIED.

[2]  Rosalind W. Picard,et al.  Automated Posture Analysis for Detecting Learner's Interest Level , 2003, 2003 Conference on Computer Vision and Pattern Recognition Workshop.

[3]  Mark C. Coulson.  Attributing Emotion to Static Body Postures: Recognition Accuracy, Confusions, and Viewpoint Dependence , 2004 .

[4]  Amy M. Witherspoon,et al.  Detection of Emotions during Learning with AutoTutor , 2006 .

[5]  Diane J. Litman,et al.  The relative impact of student affect on performance models in a spoken dialogue tutoring system , 2008, User Modeling and User-Adapted Interaction.

[6]  J. Montepare,et al.  The Use of Body Movements and Gestures as Cues to Emotions in Younger and Older Adults , 1999 .

[7]  P. Ekman,et al.  Nonverbal Leakage and Clues to Deception †. , 1969, Psychiatry.

[8]  Ashish Kapoor,et al.  Multimodal affect recognition in learning environments , 2005, ACM Multimedia.

[9]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.

[10]  W PicardRosalind,et al.  Toward an Affect-Sensitive AutoTutor , 2007 .

[11]  L. Rothkrantz,et al.  Toward an affect-sensitive multimodal human-computer interaction , 2003, Proc. IEEE.

[12]  N. Nagelkerke,et al.  A note on a general definition of the coefficient of determination , 1991 .

[13]  K. Scherer,et al.  Automated Analysis of Body Movement in Emotionally Expressive Piano Performances , 2008 .

[14]  D. K. Meyer,et al.  Re-conceptualizing Emotion and Motivation to Learn in Classroom Contexts , 2006 .

[15]  Arthur C. Graesser,et al.  Better to be frustrated than bored: The incidence, persistence, and impact of learners' cognitive-affective states during interactions with three different computer-based learning environments , 2010, Int. J. Hum. Comput. Stud..

[16]  N. A. Bernshteĭn.  The co-ordination and regulation of movements , 1967 .

[17]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[19]  J. Russell.  Core affect and the psychological construction of emotion. , 2003, Psychological review.

[20]  Arthur C. Graesser,et al.  Toward an Affect-Sensitive AutoTutor , 2007, IEEE Intelligent Systems.

[21]  BakerRyan S. J. d.,et al.  Better to be frustrated than bored , 2010 .

[22]  James C. Lester,et al.  Modeling self-efficacy in intelligent tutoring systems: An inductive approach , 2008, User Modeling and User-Adapted Interaction.

[23]  B. Mesquita,et al.  The experience of emotion. , 2007, Annual review of psychology.

[24]  Scotty D. Craig,et al.  Affect and learning: An exploratory look into the role of affect in learning with AutoTutor , 2004 .

[25]  Arthur C. Graesser,et al.  Emotions and Learning with AutoTutor , 2007, AIED.

[26]  Cristina Conati,et al.  Empirically building and evaluating a probabilistic model of user affect , 2009, User Modeling and User-Adapted Interaction.

[27]  Arthur C. Graesser,et al.  AUTOMATIC DETECTION OF LEARNER'S AFFECT FROM GROSS BODY LANGUAGE , 2009, Appl. Artif. Intell..

[28]  Arthur C. Graesser,et al.  Question asking and eye tracking during cognitive disequilibrium: Comprehending illustrated texts on devices when the devices break down , 2005, Memory & cognition.

[29]  Heather H. Mitchell,et al.  AutoTutor: A tutor with dialogue in natural language , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.