Sentic Avatar: Multimodal Affective Conversational Agent with Common Sense

The capability of perceiving and expressing emotions through different modalities is a key issue for the enhancement of human-computer interaction. In this paper we present a novel architecture for the development of intelligent multimodal affective interfaces. It is based on the integration of Sentic Computing, a new opinion mining and sentiment analysis paradigm based on AI and Semantic Web techniques, with a facial emotional classifier and Maxine, a powerful multimodal animation engine for managing virtual agents and 3D scenarios. One of the main distinguishing features of the system is that it does not simply perform emotional classification in terms of a set of discrete emotional labels but it operates in a continuous 2D emotional space, enabling the integration of the different affective extraction modules in a simple and scalable way.

[1]  Jonathan Klein,et al.  Frustrating the user on purpose: using biosignals in a pilot study to detect the user's emotional state , 1998, CHI Conference Summary.

[2]  Catherine Havasi,et al.  ConceptNet 3 : a Flexible , Multilingual Semantic Network for Common Sense Knowledge , 2007 .

[3]  Loïc Kessous,et al.  Modeling Naturalistic Affective States Via Facial, Vocal, and Bodily Expressions Recognition , 2007, Artifical Intelligence for Human Computing.

[4]  Tejashri Inadarchand Jain,et al.  Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis , 2010 .

[5]  Piyush Kumar,et al.  Minimum-Volume Enclosing Ellipsoids and Core Sets , 2005 .

[6]  Margherita Pagani.  Encyclopedia of Multimedia Technology and Networking , 2008 .

[7]  Maja Pantic,et al.  Audiovisual discrimination between laughter and speech , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[8]  S. Voloshynovskiy,et al.  Brain-computer interaction research at the computer vision and multimedia laboratory, University of Geneva , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[9]  Carlo Strapparava,et al.  WordNet Affect: an Affective Extension of WordNet , 2004, LREC.

[10]  Rosalind W. Picard.  Affective Computing , 1997 .

[11]  P. Wilson,et al.  The Nature of Emotions , 2012 .

[12]  Maja Pantic,et al.  Web-based database for facial expression analysis , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[13]  Ananth N. Iyer,et al.  Emotion Detection From Infant Facial Expressions And Cries , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[14]  C. Elliott.  The affective reasoner: a process model of emotions in a multi-agent system , 1992 .

[15]  E. Cambria,et al.  AffectiveSpace: Blending Common Sense and Affective Knowledge to Perform Emotive Reasoning , 2009 .

[16]  David A. van Leeuwen,et al.  Automatic discrimination between laughter and speech , 2007, Speech Commun..

[17]  Bing Liu,et al.  Mining Opinion Features in Customer Reviews , 2004, AAAI.

[18]  N. Ambady,et al.  Thin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis. , 1992 .

[19]  Erik Cambria,et al.  Sentic Computing: Exploitation of Common Sense for the Development of Emotion-Sensitive Systems , 2009, COST 2102 Training School.

[20]  Erik Cambria,et al.  Sentic Computing for social media marketing , 2012, Multimedia Tools and Applications.

[21]  Andrew Ortony,et al.  The Cognitive Structure of Emotions , 1988 .

[22]  Luis Mateus Rocha,et al.  Singular value decomposition and principal component analysis , 2003 .

[23]  Hatice Gunes,et al.  How to distinguish posed from spontaneous smiles using geometric features , 2007, ICMI '07.

[24]  R. Plutchik.  Emotion, a psychoevolutionary synthesis , 1980 .

[25]  Ophir Frieder,et al.  Repeatable evaluation of search services in dynamic environments , 2007, TOIS.

[26]  Claire Cardie,et al.  Annotating Expressions of Opinions and Emotions in Language , 2005, Lang. Resour. Evaluation.

[27]  Cynthia Whissell,et al.  THE DICTIONARY OF AFFECT IN LANGUAGE , 1989 .

[28]  Alex Pentland,et al.  Artifical Intelligence for Human Computing, ICMI 2006 and IJCAI 2007 International Workshops, Banff, Canada, November 3, 2006, Hyderabad, India, January 6, 2007, Revised Seleced and Invited Papers , 2007, Artifical Intelligence for Human Computing.

[29]  T. Dalgleish,et al.  Handbook of cognition and emotion , 1999 .

[30]  Hatice Gunes,et al.  Bi-modal emotion recognition from expressive face and body gestures , 2007, J. Netw. Comput. Appl..

[31]  Swapna Somasundaran,et al.  Discourse Level Opinion Interpretation , 2008, COLING.

[32]  Bo Pang,et al.  Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with Respect to Rating Scales , 2005, ACL.

[33]  David H. Evans,et al.  Detection of cough signals in continuous audio recordings using hidden Markov models , 2006, IEEE Transactions on Biomedical Engineering.

[34]  Kwee-Bo Sim,et al.  Emotion Recognition Method Based on Multimodal Sensor Fusion Algorithm , 2008, Int. J. Fuzzy Log. Intell. Syst..

[35]  E. Vesterinen,et al.  Affective Computing , 2009, Encyclopedia of Biometrics.

[36]  Shaogang Gong,et al.  Beyond Facial Expressions: Learning Human Emotion from Body Gestures , 2007, BMVC.

[37]  Hatice Gunes,et al.  From the Lab to the real world: affect recognition using multiple cues and modalities , 2008 .

[38]  Anna Esposito,et al.  Biometric ID Management and Multimodal Communication, Joint COST 2101 and 2102 International Conference, BioID_MultiComm 2009, Madrid, Spain, September 16-18, 2009. Proceedings , 2009, COST 2101/2102 Conference.

[39]  Werner Dubitzky,et al.  A Practical Approach to Microarray Data Analysis , 2003, Springer US.

[40]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[41]  Ian Witten,et al.  Data Mining , 2000 .

[42]  Anton Nijholt,et al.  Development of Multimodal Interfaces: Active Listening and Synchrony, Second COST 2102 International Training School, Dublin, Ireland, March 23-27, 2009, Revised Selected Papers , 2010, COST 2102 Training School.

[43]  Francisco J. Serón,et al.  Maxine: A platform for embodied animated agents , 2008, Comput. Graph..

[44]  Janyce Wiebe,et al.  Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis , 2005, HLT.

[45]  Volume Assp,et al.  ACOUSTICS. SPEECH. AND SIGNAL PROCESSING , 1983 .

[46]  Ken Perlin,et al.  A platform for affective agent research , 2004 .

[47]  Erik Cambria,et al.  Common Sense Computing: From the Society of Mind to Digital Intuition and beyond , 2009, COST 2101/2102 Conference.

[48]  Francisco J. Serón,et al.  Real-Time Facial Expression Recognition for Natural Interaction , 2007, IbPRIA.

[49]  Antonio Camurri,et al.  Analysis of Expressive Gesture: The EyesWeb Expressive Gesture Processing Library , 2003, Gesture Workshop.

[50]  Hsinchun Chen,et al.  Sentiment analysis in multiple languages: Feature selection for opinion classification in Web forums , 2008, TOIS.

[51]  Henry Lieberman,et al.  Digital Intuition: Applying Common Sense Using Dimensionality Reduction , 2009, IEEE Intelligent Systems.