Facial Expressions of Appraisals Displayed by a Virtual Storyteller for Children

In this paper, we present a framework for an expressive virtual storyteller for children. Our virtual storyteller displays facial expressions of appraisals related to story events. The facial expressions are animated jointly with deictic gestures towards graphical elements representing story events. We describe a preliminary study that we conducted with 23 children. We discuss the impact of facial expressions of appraisals on children’s memorization of story events, their perception of characters’ appraisals, their subjective perception of the virtual storyteller and more generally how emotion combines with joint attention.

[1]  Sara Bernardini,et al.  ECHOES: An intelligent serious game for fostering social communication in children with autism , 2014, Inf. Sci..

[2]  Marina Fridin,et al.  Storytelling by a kindergarten social assistive robot: A tool for constructive learning in preschool education , 2014, Comput. Educ..

[3]  Stacy Marsella,et al.  EMA: A process model of appraisal dynamics , 2009, Cognitive Systems Research.

[4]  K. Scherer.  Appraisal considered as a process of multilevel sequential checking. , 2001 .

[5]  Jean-Claude Martin,et al.  Modeling facial signs of appraisal during interaction: impact on users' perception and behavior , 2014, AAMAS.

[6]  Dirk Heylen,et al.  Generating expressive speech for storytelling applications , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[7]  Mariët Theune,et al.  Acting, Playing, or Talking about the Story: An Annotation Scheme for Communication during Interactive Digital Storytelling , 2013, ICIDS.

[8]  Lee Sproull,et al.  Using a human face in an interface , 1994, CHI '94.

[9]  L. Kiser,et al.  Who Are We, But for the Stories We Tell: Family Stories and Healing. , 2010, Psychological trauma : theory, research, practice and policy.

[10]  Stéphanie Buisine,et al.  Empirical investigation of the temporal relations between speech and facial expressions of emotion , 2009, Journal on Multimodal User Interfaces.

[11]  P. Petta,et al.  Computational models of emotion , 2010 .

[12]  Catherine Pelachaud,et al.  From Greta's mind to her face: modelling the dynamics of affective states in a conversational embodied agent , 2003, Int. J. Hum. Comput. Stud..

[13]  Ana Paiva,et al.  Papous: The Virtual Storyteller , 2001, IVA.

[14]  Magalie Ochs,et al.  Modélisation, formalisation et mise en œuvre d'un agent rationnel dialoguant émotionnel empathique , 2007 .

[15]  Catherine Pelachaud,et al.  Design and implementation of an expressive gesture model for a humanoid robot , 2011, 2011 11th IEEE-RAS International Conference on Humanoid Robots.

[16]  Tibor Bosse,et al.  Incorporating Emotion Regulation into Virtual Stories , 2007, IVA.

[17]  J. Bruner,et al.  The capacity for joint visual attention in the infant , 1975, Nature.

[18]  J. Bruner.  The Narrative Construction of Reality , 1991, Critical Inquiry.

[19]  Russell Beale,et al.  Affective interaction: How emotional agents affect users , 2009, Int. J. Hum. Comput. Stud..

[20]  Marc Cavazza,et al.  Emotional input for character-based interactive storytelling , 2009, AAMAS.

[21]  Radoslaw Niewiadomski,et al.  How a Virtual Agent Should Smile? - Morphological and Dynamic Characteristics of Virtual Agent's Smiles , 2010, IVA.

[22]  Dana Kulic,et al.  Measurement Instruments for the Anthropomorphism, Animacy, Likeability, Perceived Intelligence, and Perceived Safety of Robots , 2009, Int. J. Soc. Robotics.

[23]  Sara Miller,et al.  The Power of Story: Using Storytelling to Improve Literacy Learning , 2008 .

[24]  Marina Umaschi Bers,et al.  Interactive storytelling systems for children: using technology to explore language and identity , 1999 .

[25]  Nicole C. Krämer,et al.  Social Effects of Virtual Assistants. A Review of Empirical Results with Regard to Communication , 2008, IVA.