Animated Agents' Facial Emotions: Does the Agent Design Make a Difference?

The paper reports ongoing research toward the design of multimodal affective pedagogical agents that are effective for different types of learners and applications. In particular, the work reported in the paper investigated the extent to which the type of character design (realistic versus stylized) affects students’ perception of an animated agent’s facial emotions, and whether the effects are moderated by learner characteristics (e.g. gender). Eighty-two participants viewed 10 animation clips featuring a stylized character exhibiting 5 different emotions, e.g. happiness, sadness, fear, surprise and anger (2 clips per emotion), and 10 clips featuring a realistic character portraying the same emotional states. The participants were asked to name the emotions and rate their sincerity, intensity, and typicality. The results indicated that for recognition, participants were slightly more likely to recognize the emotions displayed by the stylized agent, although the difference was not statistically significant. The stylized agent was on average rated significantly higher for facial emotion intensity, whereas the differences in ratings for typicality and sincerity across all emotions were not statistically significant. A significant difference in ratings was shown in regard to sadness (within typicality), happiness (within sincerity), fear, anger, sadness and happiness (within intensity) with the stylized agent rated higher. Gender was not a significant correlate across all emotions or for individual emotions.

[1]  Sophie Jörg,et al.  Evaluating the emotional content of human motions on real and virtual characters , 2008, APGV '08.

[2]  Peter Robinson,et al.  How anthropomorphism affects empathy toward robots , 2009, 2009 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[3]  Ipke Wachsmuth,et al.  Modeling primary and secondary emotions for a believable communication agent , 2006 .

[4]  Daniel Thalmann,et al.  Emotional face expression profiles supported by virtual human ontology , 2006, Comput. Animat. Virtual Worlds.

[5]  A. D. Manning,et al.  Understanding Comics: The Invisible Art , 1993 .

[6]  Trent W. Lewis,et al.  PETA – a Pedagogical Embodied Teaching Agent , 2008 .

[7]  R. Mayer,et al.  An embodiment effect in computer-based learning with animated pedagogical agents. , 2012, Journal of experimental psychology. Applied.

[8]  R. Moreno,et al.  Students' choice of animated pedagogical agents in science learning: A test of the similarity-attraction hypothesis on gender and ethnicity , 2006 .

[9]  W. Lewis Johnson,et al.  Steve: an animated pedagogical agent for procedural training in virtual environments , 1997, SGAR.

[10]  R. Mayer,et al.  Interactive Multimodal Learning Environments , 2007 .

[11]  Noah L. Schroeder,et al.  How Effective are Pedagogical Agents for Learning? A Meta-Analytic Review , 2013 .

[12]  Arthur C. Graesser,et al.  AutoTutor and affective autotutor: Learning by talking with cognitively and emotionally intelligent computers that talk back , 2012, TIIS.

[13]  Heinrich H. Bülthoff,et al.  Evaluating the perceptual realism of animated facial expressions , 2008, TAP.

[14]  P. Ekman,et al.  Unmasking the Face: A Guide to Recognizing Emotions From Facial Expressions , 1975 .

[15]  Jean-Claude Martin,et al.  Impact of Expressive Wrinkles on Perception of a Virtual Character's Facial Expressions of Emotions , 2009, IVA.

[16]  F. Thomas,et al.  The illusion of life : Disney animation , 1981 .

[17]  Hans-Peter Seidel,et al.  Mixed feelings: expression of non-basic emotions in a muscle-based talking head , 2005, Virtual Reality.

[18]  Ning Wang,et al.  The politeness effect: Pedagogical agents and learning outcomes , 2008, Int. J. Hum. Comput. Stud..

[19]  Lawrence W. Stark,et al.  The Effects of Pictorial Realism, Delay of Visual Feedback, and Observer Interactivity on the Subjective Sense of Presence , 1996, Presence: Teleoperators & Virtual Environments.

[20]  Yanghee Kim,et al.  Gendered socialization with an embodied agent: Creating a social and affable mathematics learning environment for middle-grade females , 2013 .

[21]  Takaaki Kuratate,et al.  Making a thinking-talking head , 2007, AVSP.

[22]  Soyoung Kim,et al.  Designing nonverbal communication for pedagogical agents: When less is more , 2009, Comput. Hum. Behav..

[23]  K. Scherer Appraisal considered as a process of multilevel sequential checking. , 2001 .

[24]  Nicoletta Adamo-Villani,et al.  Keyboard encoding of facial expressions , 2004 .

[25]  Huay Lit Woo,et al.  Designing multimedia learning environments using animated pedagogical agents: factors and issues , 2009, J. Comput. Assist. Learn..

[26]  Jessica K. Hodgins,et al.  Perceptual effects of damped and exaggerated facial motion in animated characters , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[27]  Arthur C. Graesser,et al.  Adaptive Technologies for Training and Education: Emotions during Learning with AutoTutor , 2012 .

[28]  Claire Dormann,et al.  Embodied Conversational Agents on a Common Ground , 2004, From Brows to Trust.

[29]  Dongsong Zhang,et al.  Mobile personal information management agent: Supporting natural language interface and application integration , 2012, Inf. Process. Manag..

[30]  Rinat B. Rosenberg-Kima,et al.  Interface agents as social models for female students: The effects of agent visual presence and appearance on female students' attitudes and beliefs , 2008, Comput. Hum. Behav..

[31]  Dion Hoe-Lian Goh,et al.  Affect in Embodied Pedagogical Agents , 2015 .

[32]  Nicoletta Adamo-Villani,et al.  Two Novel Technologies for Accessible Math and Science Education , 2008, IEEE MultiMedia.

[33]  Christine L. Lisetti,et al.  MAUI: a multimodal affective user interface , 2002, MULTIMEDIA '02.

[34]  Yanghee Kim,et al.  The impact of learner attributes and learner choice in an agent-based environment , 2011, Comput. Educ..

[35]  Yanghee Kim,et al.  Pedagogical Agents as Social Models to Influence Learner Attitudes , 2007 .

[36]  Catherine Pelachaud,et al.  Modelling multimodal expression of emotion in a virtual agent , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[37]  Marco Paleari Psychologically grounded avatars expressions , 2006 .

[38]  Jeffrey Holmes,et al.  Designing agents to support learning by explaining , 2007, Comput. Educ..

[39]  K. Cissell A Study Of The Effects Of Computer Animated Character Body Style On Perception Of Facial Expression , 2013 .

[40]  James C. Lester,et al.  Achieving Affective Impact: Visual Emotive Communication in Lifelike Pedagogical Agents , 1999 .

[41]  Mark H. Johnson,et al.  Oxford Handbook of Face Perception , 2011 .

[42]  Ralph Adolphs,et al.  Perception and Emotion , 2006 .

[43]  Steffi Domagk,et al.  Do Pedagogical Agents Facilitate Learner Motivation and Learning Outcomes? , 2010, J. Media Psychol. Theor. Methods Appl..

[44]  James C. Lester,et al.  Lifelike Pedagogical Agents for Mixed-initiative Problem Solving in Constructivist Learning Environments , 2004, User Modeling and User-Adapted Interaction.

[45]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[46]  P. Ekman,et al.  Facial action coding system , 2019 .

[47]  James C. Lester,et al.  The persona effect: affective impact of animated pedagogical agents , 1997, CHI.

[48]  Kallirroi Georgila,et al.  SimSensei Demonstration: A Perceptive Virtual Human Interviewer for Healthcare Applications , 2015, AAAI.

[49]  Igor S. Pandzic,et al.  MPEG-4 Facial Animation , 2002 .

[50]  Yanghee Kim,et al.  Pedagogical agents as learning companions: the impact of agent emotion and gender , 2007, J. Comput. Assist. Learn..

[51]  Gloria Willcox,et al.  The Feeling Wheel A Tool for Expanding Awareness of Emotions and Increasing Spontaneity and Intimacy , 1982 .

[52]  R. Mayer The Cambridge Handbook of Multimedia Learning: Principles Based on Social Cues in Multimedia Learning: Personalization, Voice, Image, and Embodiment Principles , 2014 .

[53]  C. Nass,et al.  Truth is beauty: researching embodied conversational agents , 2001 .

[54]  Nicoletta Adamo-Villani,et al.  Does Character's Visual Style Affect Viewer's Perception of Signing Avatars? , 2015, eLEOT.