Emotional agents - state of the art and applications

last decade, intensive research on emotional intelligence has advanced significantly from its theoretical basis, analytical studies and processing technology to exploratory applications in a wide range of real-life domains. This paper brings new insights in the field of emotional, intelligent software agents. The first part is devoted to an overview of the state-of-the-art in emotional intelligence research with emphasis on emotional agents. A wide range of applications in different areas like modeling emotional agents, aspects of learning in emotional environments, interactive emotional systems and so on are presented. After that we suggest a systematic order of research steps with the idea of proposing an adequate framework for several possible real-life applications of emotional agents. We recognize that it is necessary to apply specific methods for dynamic data analysis in order to identify and discover new knowledge from available emotional information and data sets. The last part of the paper discusses research activities for designing an agent-based architecture, in which agents are capable of reasoning about and displaying some kind of emotions based on emotions detected in human speech, as well as online documents. [Projekat Ministarstva nauke Republike Srbije, br. OI174023: Intelligent techniques and their integration into wide-spectrum decision support]

[1]  Sung Bum Pan,et al.  Long distance face recognition for enhanced performance of internet of things service interface , 2014, Comput. Sci. Inf. Syst..

[2]  Carlo Ratti,et al.  An Affective Intelligent Driving Agent: Driver's Trajectory and Activities Prediction , 2009, 2009 IEEE 70th Vehicular Technology Conference Fall.

[3]  Constantine Kotropoulos,et al.  Emotional speech recognition: Resources, features, and methods , 2006, Speech Commun..

[4]  Cecilia Ovesdotter Alm,et al.  Emotions from Text: Machine Learning for Text-based Emotion Prediction , 2005, HLT.

[5]  Shrikanth S. Narayanan,et al.  Toward detecting emotions in spoken dialogs , 2005, IEEE Transactions on Speech and Audio Processing.

[6]  Chih-Wei Chang,et al.  Design an Interactive Agent by Multi-Empathic Strategies to Reduce Student's Negative Emotion , 2008, 2008 Eighth IEEE International Conference on Advanced Learning Technologies.

[7]  Jeremy V. Pitt,et al.  An Affective Anticipatory Agent Architecture , 2011, 2011 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology.

[8]  Mirjana Ivanovic,et al.  Agent-Based Distributed Computing for Dynamic Networks , 2014, Inf. Technol. Control..

[9]  Zoran Budimac,et al.  Metadata harvesting learning resources - an agent-oriented approach , 2011, 15th International Conference on System Theory, Control and Computing.

[10]  Ing-Marie Jonsson,et al.  Performance Analysis of Acoustic Emotion Recognition for In-Car Conversational Interfaces , 2007, HCI.

[11]  Roger K. Moore.  A Bayesian explanation of the ‘Uncanny Valley’ effect and related psychological phenomena , 2012, Scientific Reports.

[12]  Shuji Hashimoto,et al.  Intelligent agent system for human-robot interaction through artificial emotion , 1998, SMC'98 Conference Proceedings. 1998 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.98CH36218).

[13]  Minetada Osano,et al.  A Multi-agent Based Interactive System Towards Child’s Emotion Performances Quantified Through Affective Body Gestures , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[14]  Tibor Bosse,et al.  Development of Virtual Agents with a Theory of Emotion Regulation , 2008, 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology.

[15]  Majed Al-Shawa.  Eliciting agents' preferences from contextual knowledge about their goals, constraints, priorities, emotions and attitudes using the Constrained Rationality framework , 2011, 2011 IEEE International Conference on Systems, Man, and Cybernetics.

[16]  Igor S. Pandžić,et al.  Facial Gestures: Taxonomy and Application of Nonverbal, Nonemotional Facial Displays for Embodied Conversational Agents , 2007 .

[17]  Kijoon Chae,et al.  Pairwise and group key setup mechanism for secure machine-to-machine communication , 2014, Comput. Sci. Inf. Syst..

[18]  Zhiliang Wang,et al.  An Emotional Intelligent E-learning System Based on Mobile Agent Technology , 2009, 2009 International Conference on Computer Engineering and Technology.

[19]  Mohammad Mansour Riahi Kashani,et al.  A model of intelligent agent system using horror emotion , 2010, 2010 International Conference on Education and Management Technology.

[20]  Jan Treur,et al.  Cognitive and Biological Agent Models for Emotion Reading , 2008, 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology.

[21]  Ryan Shaun Joazeiro de Baker,et al.  The Effects of an Interactive Software Agent on Student Affective Dynamics while Using ;an Intelligent Tutoring System , 2012, IEEE Transactions on Affective Computing.

[22]  Costin Badica,et al.  Relating the Opinion Holder and the Review Accuracy in Sentiment Analysis of Tourist Reviews , 2014, KSEM.

[23]  Scott A. King,et al.  Creating Emotional Speech for Conversational Agents , 2011, 2011 Workshop on Digital Media and Digital Content Management.

[24]  Ryohei Nakatsu,et al.  Life-like communication agent-emotion sensing character "MIC" and feeling session character "MUSE" , 1996, Proceedings of the Third IEEE International Conference on Multimedia Computing and Systems.

[25]  Wu Li,et al.  Speech Emotion Recognition in E-learning System Based on Affective Computing , 2007, Third International Conference on Natural Computation (ICNC 2007).

[26]  Mann Oo. Hay.  Emotion recognition in human-computer interaction , 2012 .

[27]  Piotr J. Gmytrasiewicz,et al.  Using Decision Theory to Formalize Emotions for Multi-Agent System Applications: Preliminary Report , 2000 .

[28]  Toshio Fukuda,et al.  Individuality of agent with emotional algorithm , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[29]  Zhiliang Wang,et al.  The research of interactive system of emotional robot based on multi-agent , 2006, 2006 1st International Symposium on Systems and Control in Aerospace and Astronautics.

[30]  María Malfaz,et al.  A New Approach to Modeling Emotions and Their Use on a Decision-Making System for Artificial Agents , 2012, IEEE Transactions on Affective Computing.

[31]  H. Prendinger,et al.  Affective and social behavior in animated agents , 2001, Proceedings Joint 9th IFSA World Congress and 20th NAFIPS International Conference (Cat. No. 01TH8569).

[32]  Guillaume Hutzler,et al.  Emotion Modeling for Intelligent Agents - Towards a Unifying Framework , 2011, 2011 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology.

[33]  G. Clarebout,et al.  Do pedagogical agents make a difference to student motivation and learning , 2011 .

[34]  J. G. Taylor,et al.  Emotion recognition in human-computer interaction , 2005, Neural Networks.

[35]  Peter W. McOwan,et al.  A real-time automated system for the recognition of human facial expressions , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[36]  Philippe Fournier-Viger,et al.  How emotional mechanism helps episodic learning in a cognitive agent , 2009, 2009 IEEE Symposium on Intelligent Agents.

[37]  Jose Jesus Castro-Schez,et al.  Using an Emotional Intelligent Agent to Support Customers' Searches Interactively in e-Marketplaces , 2010, 2010 22nd IEEE International Conference on Tools with Artificial Intelligence.

[38]  A. Camurri,et al.  An Architecture for Emotional Agents , 1998, IEEE Multim..

[39]  Thomas P. Hartley,et al.  Emotions for intelligent agents in crisis management simulations: A survey , 2012, 2012 17th International Conference on Computer Games (CGAMES).

[40]  Konstantina Chatzara,et al.  Students Attitude and Learning Effectiveness of Emotional Agents , 2010, 2010 10th IEEE International Conference on Advanced Learning Technologies.

[41]  Tim Polzehl,et al.  Emotion detection in dialog systems: Applications, strategies and challenges , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[42]  Shashidhar G. Koolagudi,et al.  Emotion recognition from speech: a review , 2012, International Journal of Speech Technology.

[43]  Rosalind W. Picard.  Affective computing: (526112012-054) , 1997 .

[44]  Pierre Dumouchel,et al.  Anchor Models for Emotion Recognition from Speech , 2013, IEEE Transactions on Affective Computing.

[45]  Maiga Chang,et al.  Foundation analysis of emotion model for designing learning companion agent , 2004, IEEE International Conference on Advanced Learning Technologies, 2004. Proceedings..

[46]  R. Gunderman,et al.  Emotional intelligence. , 2011, Journal of the American College of Radiology : JACR.

[47]  Anastasios A. Economides,et al.  Affective Learning: Empathetic Agents with Emotional Facial and Tone of Voice Expressions , 2012, IEEE Transactions on Affective Computing.

[48]  A. Crespo,et al.  Behavioural Affective Modulation of Service Robotic Agents , 2006, IECON 2006 - 32nd Annual Conference on IEEE Industrial Electronics.

[49]  Ramón López-Cózar,et al.  Influence of contextual information in emotion annotation for spoken dialogue systems , 2008, Speech Commun..

[50]  Mehryar Mohri,et al.  A comparison of classifiers for detecting emotion from speech , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[51]  Alfons Crespo,et al.  Multicore Parallel Implementation of Agent Emotional Processes , 2013, 2013 12th IEEE International Conference on Trust, Security and Privacy in Computing and Communications.

[52]  Xianwei Lai,et al.  A Theoretical Framework of Rational and Emotional Agent for Ubiquitous Computing , 2006, 16th International Conference on Artificial Reality and Telexistence--Workshops (ICAT'06).

[53]  Fabrice Guillet,et al.  A system of emotional agents for decision-support , 2005, IEEE/WIC/ACM International Conference on Intelligent Agent Technology.

[54]  Chung-Hsien Wu,et al.  Emotion recognition from text using semantic labels and separable mixture models , 2006, TALIP.

[55]  Knut Hartmann,et al.  Emotional Agents for Interactive Environments , 2006, Fourth International Conference on Creating, Connecting and Collaborating through Computing (C5'06).

[56]  Sara Owsley Sood,et al.  Emotional computation in artificial intelligence education , 2008, AAAI 2008.

[57]  Kostas Karpouzis,et al.  Emotion recognition through facial expression analysis based on a neurofuzzy network , 2005, Neural Networks.

[58]  Ze-Jing Chuang,et al.  Multi-Modal Emotion Recognition from Speech and Text , 2004, ROCLING/IJCLCLP.

[59]  Emiliano Lorini,et al.  A New Look at the Semantics and Optimization Methods of CP-Networks , 2003, IJCAI.

[60]  Carlo Strapparava,et al.  Learning to identify emotions in text , 2008, SAC '08.

[61]  Boon Tan,et al.  The Effect of Gender-Emotion Stereotypes in Communicating Emotion through Affective Agents , 2006, 2006 International Conference on Computational Inteligence for Modelling Control and Automation and International Conference on Intelligent Agents Web Technologies and International Commerce (CIMCA'06).

[62]  Zoran Budimac,et al.  An overview of ontologies and data resources in medical domains , 2014, Expert Syst. Appl..

[63]  Dong-Soo Kwon,et al.  Episodic memory system of affective agent with emotion for long-term human-robot interaction , 2013, 2013 10th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI).

[64]  Zoran Budimac,et al.  Emotional Intelligence and Agents: Survey and Possible Applications , 2014, WIMS '14.

[65]  Yoshifumi Kitamura,et al.  Modulating reinforcement-learning parameters using agent emotions , 2012, The 6th International Conference on Soft Computing and Intelligent Systems, and The 13th International Symposium on Advanced Intelligence Systems.

[66]  Kasia Muldner,et al.  The Impact of Animated Pedagogical Agents on Girls' and Boys' Emotions, Attitudes, Behaviors and Learning , 2011, 2011 IEEE 11th International Conference on Advanced Learning Technologies.

[67]  Alexander Wendt,et al.  A psychoanalytically-inspired motivational and emotional system for autonomous agents , 2013, IECON 2013 - 39th Annual Conference of the IEEE Industrial Electronics Society.

[68]  A. Crespo,et al.  Real-time robotic agent based on emotions , 2004, Proceedings World Automation Congress, 2004..

[69]  Baher Abdulhai,et al.  Emotional temporal difference Q-learning signals in multi-agent system cooperation: real case studies , 2013 .

[70]  Mehdi Dastani,et al.  A formal model of emotion triggers: an approach for BDI agents , 2011, Synthese.

[71]  Domitile Lourdeaux,et al.  Personality, Emotions and Physiology in a BDI Agent Architecture: The PEP -≫ BDI Model , 2009, 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology.

[72]  T. Fukuda,et al.  Fuzzy emotion interpolation system for emotional autonomous agents , 2002, Proceedings of the 41st SICE Annual Conference. SICE 2002..

[73]  Mirjana Ivanovic,et al.  The Influence of Global Constraints on Similarity Measures for Time-Series Databases , 2011, Knowl. Based Syst..

[74]  Mitsuru Ishizuka,et al.  ASNA: An Intelligent Agent for Retrieving and Classifying News on the Basis of Emotion-Affinity , 2006, 2006 International Conference on Computational Inteligence for Modelling Control and Automation and International Conference on Intelligent Agents Web Technologies and International Commerce (CIMCA'06).

[75]  Dongrong Xu,et al.  Emotional Intelligence System for Ubiquitous Smart Foreign Language Education Based on Neural Mechanism , 2014 .

[76]  E. Vesterinen,et al.  Affective Computing , 2009, Encyclopedia of Biometrics.

[77]  Jan Treur,et al.  An ambient agent model for group emotion support , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[78]  Yan-You Chen,et al.  Application of Educational Emotion Inference via Speech and Agent Interaction , 2010, 2010 Third IEEE International Conference on Digital Game and Intelligent Toy Enhanced Learning.

[79]  Felix D. Schönbrodt,et al.  The Challenge of Constructing Psychologically Believable Agents , 2011, J. Media Psychol. Theor. Methods Appl..

[80]  Matthias Scheutz,et al.  The Affect Dilemma for Artificial Agents: Should We Develop Affective Artificial Agents? , 2012, IEEE Transactions on Affective Computing.

[81]  Z.A. Shaikh,et al.  An Approach Towards Emotion Estimation During Chat Sessions Using Software Agents , 2007, 2007 Innovations in Information Technologies (IIT).

[82]  N.G. Aghaei,et al.  The structure of personality-based emotional decision making in robotic rescue agent , 2008, 2008 IEEE/ASME International Conference on Advanced Intelligent Mechatronics.

[83]  Joel R. Tetreault,et al.  Using system and user performance features to improve emotion detection in spoken tutoring dialogs , 2006, INTERSPEECH.

[84]  Peter Ohler,et al.  Multimodal Emotion Expressions of Virtual Agents, Mimic and Vocal Emotion Expressions and Their Effects on Emotion Recognition , 2013, 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction.

[85]  M. Seif El-Nasr,et al.  A fuzzy emotional agent for decision-making in a mobile robot , 1998, 1998 IEEE International Conference on Fuzzy Systems Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36228).

[86]  Mirjana Ivanovic,et al.  Distributed distance matrix generator based on agents , 2012, WIMS '12.

[87]  Wang Guojiang,et al.  Behavior decision model of intelligent agent based on artificial emotion , 2010, 2010 2nd International Conference on Advanced Computer Control.

[88]  Ho-Jin Choi,et al.  An Intelligent and Emotional Web Browsing Agent , 2012, 2012 International Conference on Information Science and Applications.

[89]  Ilsun You,et al.  A computer remote control system based on speech recognition technologies of mobile devices and wireless communication technologies , 2014, Comput. Sci. Inf. Syst..

[90]  Toru Yamaguchi,et al.  Cooperative works for agent robot and human using robot vision based on the model of knowledge, emotion and intention , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[91]  Ildikó Pelczer,et al.  Emotions and Interactive Agents , 2006, VECIMS.