Monitoring Affect States During Effortful Problem Solving Activities

We explored the affective states that students experienced during effortful problem solving activities. We conducted a study where 41 students solved difficult analytical reasoning problems from the Law School Admission Test. Students viewed videos of their faces and screen captures and judged their emotions from a set of 14 states (basic emotions, learning-centered emotions, and neutral) at relevant points in the problem solving process (after new problem is displayed, in the midst of problem solving, after feedback is received). The results indicated that curiosity, frustration, boredom, confusion, happiness, and anxiety were the major emotions that students experienced, while contempt, anger, sadness, fear, disgust, eureka, and surprise were rare. Follow-up analyses on the temporal dynamics of the emotions, their contextual underpinnings, and relationships to problem solving outcomes supported a general characterization of the affective dimension of problem solving. Affective states differ in: (a) their probability of occurrence as regular, routine, or sporadic emotions, (b) their temporal dynamics as persistent or random emotions, (c) their characterizations as product or process related emotions, and (d) whether they were positively or negatively related to problem solving outcomes. A synthesis of our major findings, limitations, resolutions, and implications for affect-sensitive artificial learning environments are discussed.

[1]  Sidney D'Mello,et al.  The half-life of cognitive-affective states during complex learning , 2011, Cognition & emotion.

[2]  P. V. Lange,et al.  Handbook of Theories of Social Psychology , 2011 .

[3]  R. Calvo,et al.  New Perspectives on Affect and Learning Technologies , 2011 .

[4]  Ma. Mercedes T. Rodrigo,et al.  Comparing the Incidence and Persistence of Learners’ Affect During Interactions with Different Educational Software Packages , 2011 .

[5]  Diane J. Litman,et al.  Designing and evaluating a wizarded uncertainty-adaptive spoken dialogue tutoring system , 2011, Comput. Speech Lang..

[6]  Andrew Olney,et al.  Mining Collaborative Patterns in Tutorial Dialogues , 2010, EDM 2010.

[7]  Reinhard Pekrun,et al.  Boredom in achievement settings: Exploring control–value antecedents and performance outcomes of a neglected emotion. , 2010 .

[8]  Kasia Muldner,et al.  The Effect of Motivational Learning Companions on Low Achieving Students and Students with Disabilities , 2010, Intelligent Tutoring Systems.

[9]  Arthur C. Graesser,et al.  Multimodal semi-automated affect detection from conversational cues, gross body language, and facial features , 2010, User Modeling and User-Adapted Interaction.

[10]  Arthur C. Graesser,et al.  Better to be frustrated than bored: The incidence, persistence, and impact of learners' cognitive-affective states during interactions with three different computer-based learning environments , 2010, Int. J. Hum. Comput. Stud..

[11]  G. Pourtois,et al.  The perception and categorisation of emotional stimuli: A review , 2010 .

[12]  Arthur C. Graesser,et al.  Modeling Cognitive-Affective Dynamics with Hidden Markov Models , 2010 .

[13]  A. Moors Theories of emotion causation , 2010 .

[14]  Rafael A. Calvo,et al.  Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications , 2010, IEEE Transactions on Affective Computing.

[15]  James C. Lester,et al.  Evaluating the consequences of affective feedback in intelligent tutoring systems , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[16]  Peter Robinson,et al.  Natural affect data — Collection & annotation in a learning context , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[17]  Nico H. Frijda,et al.  Emotions, individual differences and time course: Reflections , 2009 .

[18]  K. Scherer The dynamic architecture of emotion: Evidence for the component process model , 2009 .

[19]  L. F. Barrett Variety is the spice of life: A psychological construction approach to understanding variability in emotion , 2009, Cognition & emotion.

[20]  Cristina Conati,et al.  Empirically building and evaluating a probabilistic model of user affect , 2009, User Modeling and User-Adapted Interaction.

[21]  Kasia Muldner,et al.  Emotion Sensors Go To School , 2009, AIED.

[22]  Arthur C. Graesser,et al.  Responding to Learners' Cognitive-Affective States with Supportive and Shakeup Dialogues , 2009, HCI.

[23]  Arthur C. Graesser,et al.  Meta-Knowledge in Tutoring , 2009 .

[24]  A. Moors Theories of emotion causation: A review , 2009 .

[25]  S. Mann,et al.  Boredom in the lecture theatre: An investigation into the contributors, moderators and outcomes of boredom amongst university students , 2009 .

[26]  P. Silvia Looking past pleasure: Anger, confusion, disgust, pride, surprise, and other unusual aesthetic emotions. , 2009 .

[27]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Ran R. Hassin,et al.  Angry, Disgusted, or Afraid? , 2008, Psychological science.

[29]  Arthur C. Graesser,et al.  Agent Technologies Designed to Facilitate Interactive Knowledge Construction , 2008 .

[30]  Arthur C. Graesser,et al.  Self Versus Teacher Judgments of Learner Emotions During a Tutoring Session with AutoTutor , 2008, Intelligent Tutoring Systems.

[31]  Sidney K. D'Mello,et al.  What Are You Feeling? Investigating Student Affective States During Expert Human Tutoring Sessions , 2008, Intelligent Tutoring Systems.

[32]  Loïc Kessous,et al.  Emotion Recognition through Multiple Modalities: Face, Body Gesture, Speech , 2008, Affect and Emotion in Human-Computer Interaction.

[33]  A. Isen,et al.  Some ways in which positive affect influences decision making and problem solving. , 2008 .

[34]  Diane J. Litman,et al.  The relative impact of student affect on performance models in a spoken dialogue tutoring system , 2008, User Modeling and User-Adapted Interaction.

[35]  Arthur C. Graesser,et al.  Automatic detection of learner’s affect from conversational cues , 2008, User Modeling and User-Adapted Interaction.

[36]  James C. Lester,et al.  Modeling self-efficacy in intelligent tutoring systems: An inductive approach , 2008, User Modeling and User-Adapted Interaction.

[37]  K. Scherer,et al.  The World of Emotions is not Two-Dimensional , 2007, Psychological science.

[38]  C. Izard Basic Emotions, Natural Kinds, Emotion Schemas, and a New Paradigm , 2007, Perspectives on psychological science : a journal of the Association for Psychological Science.

[39]  G. Clore,et al.  How emotions inform judgment and regulate thought , 2007, Trends in Cognitive Sciences.

[40]  Winslow Burleson,et al.  Gender-Specific Approaches to Developing Emotionally Intelligent Learning Companions , 2007, IEEE Intelligent Systems.

[41]  Arthur C. Graesser,et al.  Emotions and Learning with AutoTutor , 2007, AIED.

[42]  V. Shute Focus on Formative Feedback , 2007 .

[43]  John J. B. Allen,et al.  The handbook of emotion elicitation and assessment , 2007 .

[44]  A. Graesser,et al.  Monitoring Affective Trajectories during Complex Learning , 2007 .

[45]  D. K. Meyer,et al.  Re-conceptualizing Emotion and Motivation to Learn in Classroom Contexts , 2006 .

[46]  Vincent Aleven,et al.  Toward Meta-cognitive Tutoring: A Model of Help Seeking with a Cognitive Tutor , 2006, Int. J. Artif. Intell. Educ..

[47]  L. F. Barrett Are Emotions Natural Kinds? , 2006, Perspectives on psychological science : a journal of the Association for Psychological Science.

[48]  Arthur C. Graesser,et al.  Predicting Affective States expressed through an Emote-Aloud Procedure from AutoTutor's Mixed-Initiative Dialogue , 2006, Int. J. Artif. Intell. Educ..

[49]  Amy M. Witherspoon,et al.  Detection of Emotions during Learning with AutoTutor , 2006 .

[50]  A. Efklides Metacognition and Affect: What Can Metacognitive Experiences Tell Us about the Learning Process?. , 2006 .

[51]  J. Funke,et al.  The role of emotions in complex problem solving , 2005 .

[52]  Arthur C. Graesser,et al.  Question asking and eye tracking during cognitive disequilibrium: Comprehending illustrated texts on devices when the devices break down , 2005, Memory & cognition.

[53]  Roger Azevedo,et al.  Self-Regulation of Learning with Multiple Representations in Hypermedia , 2005, AIED.

[54]  Richard S. Lazarus,et al.  The cognition–emotion debate: A bit of history. , 2005 .

[55]  Scotty D. Craig,et al.  Affect and learning: An exploratory look into the role of affect in learning with AutoTutor , 2004 .

[56]  John R. Anderson,et al.  How Should a Theory of Learning and Cognition Inform Instruction , 2004 .

[57]  A. Isen,et al.  POSITIVE AFFECT, SYSTEMATIC COGNITIVE PROCESSING, AND BEHAVIOR: TOWARD INTEGRATION OF AFFECT, COGNITION, AND MOTIVATION , 2003 .

[58]  J. Russell,et al.  Facial and vocal expressions of emotion. , 2003, Annual review of psychology.

[59]  Brent A. Olde,et al.  How does one know whether a person understands a device? The quality of the questions the person asks when the device breaks down. , 2003 .

[60]  K. VanLehn,et al.  Why Do Only Some Events Cause Learning During Human Tutoring? , 2003 .

[61]  N. Schwarz,et al.  The Psychology of Problem Solving: Feeling and Thinking: Implications for Problem Solving , 2003 .

[62]  P. Rozin,et al.  High frequency of facial expressions corresponding to confusion, concentration, and worry in an analysis of naturally occurring facial expressions of Americans. , 2003, Emotion.

[63]  J. Russell Core affect and the psychological construction of emotion. , 2003, Psychological review.

[64]  Vincent Aleven,et al.  An effective metacognitive strategy: learning by doing and explaining with a computer-based Cognitive Tutor , 2002, Cogn. Sci..

[65]  Cs Dweck,et al.  Messages that motivate: How praise molds students’ beliefs, motivation, and performance (in surprising ways). Aronson (Ed.), Improving academic achievement: Impact of psychological factors on education (pp. ). Amsterdam: Academic Press. , 2002 .

[66]  Mark R. Lepper,et al.  The wisdom of practice: Lessons learned from the study of highly effective tutors. , 2002 .

[67]  Jonathan Klein,et al.  This computer responds to user frustration: Theory, design, and results , 2002, Interact. Comput..

[68]  Rosalind W. Picard,et al.  An affective model of interplay between emotions and learning: reengineering educational pedagogy-building a learning companion , 2001, Proceedings IEEE International Conference on Advanced Learning Technologies.

[69]  Takashi Yamauchi,et al.  Learning from human tutoring , 2001, Cogn. Sci..

[70]  T. Scherer,et al.  Constraints for emotion specificity in fear and anger: the context counts. , 2001, Psychophysiology.

[71]  Philip H. Winne,et al.  Self-regulated learning viewed from models of information processing. , 2001 .

[72]  Klaus Fiedler,et al.  Affective states trigger processes of assimilation and accommodation , 2001 .

[73]  N. Kerr,et al.  Mood effects on cooperation in small groups: Does positive mood simply lead to more cooperation? , 2000 .

[74]  N. Schwarz Emotion, cognition, and decision making , 2000 .

[75]  Kurt VanLehn,et al.  Andes: A Coached Problem Solving Environment for Physics , 2000, Intelligent Tutoring Systems.

[76]  L. F. Barrett,et al.  Handbook of emotions, 2nd ed. , 2000 .

[77]  P. Pintrich,et al.  Handbook of self-regulation , 2000 .

[78]  K. Visweswaran Affective States , 1999 .

[79]  K. Trigwell,et al.  Understanding Learning and Teaching: the experience in higher education , 1999 .

[80]  David H. Jonassen,et al.  Learning with Technology: A Constructivist Perspective , 1998 .

[81]  P. Black,et al.  Assessment and Classroom Learning , 1998 .

[82]  C. J. McGrath,et al.  Effect of exchange rate return on volatility spill-over across trading regions , 2012 .

[83]  J. Biggs Enhancing teaching through constructive alignment , 1996 .

[84]  L. Steels,et al.  Grounding adaptive language games in robotic agents , 2006, AAAI 2012.

[85]  D. Jackson,et al.  Individual differences in affective and conative functions. , 1996 .

[86]  Roger Azevedo,et al.  A Meta-Analysis of the Effects of Feedback in Computer-Based Instruction , 1995 .

[87]  Klaus Fiedler,et al.  Affective States and the Influence of Activated General Knowledge , 1995 .

[88]  P. Ekman,et al.  Coherence between expressive and experiential systems in emotion , 1994 .

[89]  S. Tobias Interest, Prior Knowledge, and Learning , 1994 .

[90]  L. F. Barrett,et al.  Handbook of Emotions , 1993 .

[91]  Arne Öhman,et al.  On the automatic nature of phobic fear: conditioned electrodermal responses to masked fear-relevant stimuli. , 1993, Journal of abnormal psychology.

[92]  Edward L. Deci,et al.  Article Commentary: On the Nature and Functions of Motivation Theories , 1992 .

[93]  P. Ekman An argument for basic emotions , 1992 .

[94]  Reed W. Larson,et al.  Boredom in the Middle School Years: Blaming Schools versus Blaming Students , 1991, American Journal of Education.

[95]  Chen-Lin C. Kulik,et al.  The Instructional Effect of Feedback in Test-Like Events , 1991 .

[96]  M. Csíkszentmihályi Flow: The Psychology of Optimal Experience , 1990 .

[97]  N. Schwarz Feelings as information: Informational and motivational functions of affective states. , 1990 .

[98]  G. A. Mendelsohn,et al.  Affect grid : A single-item scale of pleasure and arousal , 1989 .

[99]  Douglas B. McLeod,et al.  Affective Issues in Mathematical Problem Solving: Some Theoretical Considerations. , 1988 .

[100]  Susanne P. Lajoie,et al.  Sherlock: A Coached Practice Environment for an Electronics Troubleshooting Job. , 1988 .

[101]  R. White,et al.  Giving Voice to High School Students: Pressure and Boredom, Ya Know What I’m Sayin’? , 1988 .

[102]  J Keenan,et al.  "Variety is the spice of life". , 1987, Dental health.

[103]  A. Isen,et al.  Positive affect facilitates creative problem solving , 1987 .

[104]  A. Brown Metacognition, executive control, self-regulation, and other more mysterious mechanisms , 1987 .

[105]  B. Weiner An attributional theory of motivation and emotion , 1986 .

[106]  A. B. Hill,et al.  Cognitive and affective aspects of boredom. , 1985, British journal of psychology.

[107]  Noel Entwistle,et al.  Styles of learning and teaching , 1981 .

[108]  G. Mandler Mind and Emotion , 1975 .

[109]  W. Robinson,et al.  Boredom at school. , 1975, The British journal of educational psychology.

[110]  F. Heider The psychology of interpersonal relations , 1958 .

[111]  L. Festinger,et al.  A Theory of Cognitive Dissonance , 2017 .