Towards Emotion Recognition in Human Computer Interaction

The recognition of human emotions by technical systems is regarded as a problem of pattern recognition. Here methods of machine learning are employed which require substantial amounts of ’emotionally labeled’ data, because model based approaches are not available. Problems of emotion recognition are discussed from this point of view, focusing on problems of data gathering and also touching upon modeling of emotions and machine learning aspects.

[1]  Björn W. Schuller,et al.  Context-sensitive multimodal emotion recognition from speech and facial expression using bidirectional LSTM modeling , 2010, INTERSPEECH.

[2]  Markus Kächele,et al.  Multiple Classifier Systems for the Classification of Audio-Visual Emotional States , 2011, ACII.

[3]  Günther Palm,et al.  Classifier Fusion Applied to Facial Expression Recognition: An Experimental Comparison , 2009, Human Centered Robot Systems, Cognition, Interaction, Technology.

[4]  Günther Palm,et al.  Classification of Time Series Utilizing Temporal and Decision Fusion , 2001, Multiple Classifier Systems.

[5]  Michelle Karg,et al.  Physiology and HRI: Recognition of over- and underchallenge , 2008, RO-MAN 2008 - The 17th IEEE International Symposium on Robot and Human Interactive Communication.

[6]  Günther Palm,et al.  Semi-supervised learning for tree-structured ensembles of RBF networks with Co-Training , 2010, Neural Networks.

[7]  Roddy Cowie,et al.  Emotional speech: Towards a new generation of databases , 2003, Speech Commun..

[8]  A. Mehrabian.  Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in Temperament , 1996 .

[9]  Zhi-Hua Zhou,et al.  Analyzing Co-training Style Algorithms , 2007, ECML.

[10]  J. Russell,et al.  Core affect, prototypical emotional episodes, and other things called emotion: dissecting the elephant. , 1999 .

[11]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[12]  Maja Pantic,et al.  The SEMAINE corpus of emotionally coloured character interactions , 2010, 2010 IEEE International Conference on Multimedia and Expo.

[13]  Friedhelm Schwenker,et al.  Experiments with Supervised Fuzzy LVQ , 2008, ANNPR.

[14]  Friedhelm Schwenker,et al.  Combining committee-based semi-supervised learning and active learning , 2010 .

[15]  Günther Palm,et al.  A Study of the Robustness of KNN Classifiers Trained Using Soft Labels , 2006, ANNPR.

[16]  Catherine Pelachaud,et al.  The HUMAINE Database , 2011 .

[17]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[18]  Günther Palm,et al.  A Novel Feature for Emotion Recognition in Voice Based Applications , 2007, ACII.

[19]  Andreas Wendemuth,et al.  Companion-Technology for Cognitive Technical Systems , 2011, KI - Künstliche Intelligenz.

[20]  Florentin Wörgötter,et al.  Temporal Sequence Learning, Prediction, and Control: A Review of Different Models and Their Relation to Biological Mechanisms , 2005, Neural Computation.

[21]  Shrikanth S. Narayanan,et al.  The Vera am Mittag German audio-visual emotional speech database , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[22]  Michael A. Arbib,et al.  Extending the mirror neuron system model, II: what did I just do? A new role for mirror neurons , 2010, Biological Cybernetics.

[23]  Elmar Nöth,et al.  From Emotion to Interaction: Lessons from Real Human-Machine-Dialogues , 2004, ADS.

[24]  Björn W. Schuller,et al.  AVEC 2011-The First International Audio/Visual Emotion Challenge , 2011, ACII.

[25]  Wolfgang Minker,et al.  Multi-Party Dialogue Corpus , 2010 .

[26]  Pierre-Yves Oudeyer,et al.  Intrinsic Motivation Systems for Autonomous Mental Development , 2007, IEEE Transactions on Evolutionary Computation.

[27]  Günther Palm,et al.  Spotting laughter in natural multiparty conversations: A comparison of automatic online and offline approaches using audiovisual data , 2012, TIIS.

[28]  Günther Palm,et al.  Multimodal Laughter Detection in Natural Discourses , 2009, Human Centered Robot Systems, Cognition, Interaction, Technology.

[29]  Friedhelm Schwenker,et al.  Decision Templates Based RBF Network for Tree-Structured Multiple Classifier Fusion , 2009, MCS.

[30]  Friedhelm Schwenker,et al.  Fuzzy-Input Fuzzy-Output One-Against-All Support Vector Machines , 2007, KES.

[31]  Rosalind W. Picard.  Affective computing: (526112012-054) , 1997 .

[32]  W. Cannon.  The James-Lange theory of emotions: a critical examination and an alternative theory. By Walter B. Cannon, 1927. , 1927, The American journal of psychology.

[33]  Andrew Ortony,et al.  The Cognitive Structure of Emotions , 1988 .

[34]  Kostas Karpouzis,et al.  The HUMAINE Database: Addressing the Collection and Annotation of Naturalistic and Induced Emotional Data , 2007, ACII.

[35]  Sascha Meudt,et al.  Developing an Expressive Speech Labeling Tool Incorporating the Temporal Characteristics of Emotion , 2010, LREC.

[36]  Rosalind W. Picard.  Affective Computing , 1997 .

[37]  Ioannis Pitas,et al.  The eNTERFACE’05 Audio-Visual Emotion Database , 2006, 22nd International Conference on Data Engineering Workshops (ICDEW'06).

[38]  Friedhelm Schwenker,et al.  Three learning phases for radial-basis-function networks , 2001, Neural Networks.

[39]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2009, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  Catharine Oertel Gen Bierbach.  On the use of multimodal cues for the prediction of involvement in spontaneous conversation , 2011 .

[41]  Günther Palm,et al.  Decision templates for the classification of bioacoustic time series , 2003, Inf. Fusion.

[42]  John G. Taylor,et al.  The interaction of attention and emotion , 2005, Neural Networks.

[43]  Zhigang Deng,et al.  Emotion recognition based on phoneme classes , 2004, INTERSPEECH.

[44]  P. Ekman,et al.  The Repertoire of Nonverbal Behavior: Categories, Origins, Usage, and Coding , 1969 .

[45]  Björn W. Schuller,et al.  Frame vs. Turn-Level: Emotion Recognition from Speech Considering Static and Dynamic Processing , 2007, ACII.

[46]  Y. Niv.  Reinforcement learning in the brain , 2009 .

[47]  Ulrich Trk.  The technical processing in smartkom data collection: a case study , 2001, INTERSPEECH.

[48]  P. Dayan,et al.  Serotonin in affective control. , 2009, Annual review of neuroscience.

[49]  M. Botvinick,et al.  Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective , 2009, Cognition.

[50]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[51]  P. Dayan,et al.  A Bayesian formulation of behavioral control , 2009, Cognition.

[52]  Günther Palm,et al.  The PIT Corpus of German Multi-Party Dialogues , 2008, LREC.

[53]  Elmar Nöth,et al.  We are not amused - but how do you know? user states in a multi-modal dialogue system , 2003, INTERSPEECH.

[54]  Günther Palm,et al.  Comparison of Multiclass SVM Decomposition Schemes for Visual Object Recognition , 2005, DAGM-Symposium.

[55]  G. Rizzolatti,et al.  Mirrors in the Brain: How Our Minds Share Actions and Emotions , 2007 .

[56]  Ayhan Demiriz,et al.  Exploiting unlabeled data in ensemble methods , 2002, KDD.

[57]  Astrid Paeschke,et al.  A database of German emotional speech , 2005, INTERSPEECH.

[58]  Yoshua Bengio,et al.  Semi-supervised Learning by Entropy Minimization , 2004, CAP.

[59]  Ludmila I. Kuncheva,et al.  Naive random subspace ensemble with linear classifiers for real-time classification of fMRI data , 2012, Pattern Recognit..

[60]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[61]  Elmar Nöth,et al.  How to find trouble in communication , 2003, Speech Commun..

[62]  Anna Esposito,et al.  The Amount of Information on Emotional States Conveyed by the Verbal and Nonverbal Channels: Some Perceptual Data , 2005, WNSP.

[63]  Friedhelm Schwenker,et al.  Classification of bioacoustic time series based on the combination of global and local decisions , 2004, Pattern Recognit..

[64]  Michael A. Arbib,et al.  Extending the mirror neuron system model, I , 2007, Biological Cybernetics.

[65]  James S. Albus,et al.  Outline for a theory of intelligence , 1991, IEEE Trans. Syst. Man Cybern..

[66]  Xin Yao,et al.  Diversity creation methods: a survey and categorisation , 2004, Inf. Fusion.

[67]  Ingo Siegert,et al.  ikannotate - A Tool for Labelling, Transcription, and Annotation of Emotionally Coloured Speech , 2011, ACII.

[68]  Nick Campbell.  Databases of Expressive Speech , 2004, J. Chin. Lang. Comput..

[69]  J. Russell.  A circumplex model of affect. , 1980 .

[70]  M. Arbib.  Action to language via the mirror neuron system , 2006 .

[71]  Dietmar F. Rösner,et al.  On the Role of the NIMITEK Corpus in Developing an Emotion Adaptive Spoken Dialogue System , 2008, LREC.

[72]  Klaus R. Scherer,et al.  Using Actor Portrayals to Systematically Study Multimodal Emotion Expression: The GEMEP Corpus , 2007, ACII.

[73]  Xiaojin Zhu,et al.  Semi-Supervised Learning Literature Survey , 2005 .

[74]  Günther Palm,et al.  Semi-supervised Facial Expressions Annotation Using Co-Training with Fast Probabilistic Tri-Class SVMs , 2010, ICANN.

[75]  Roddy Cowie,et al.  Beyond emotion archetypes: Databases for emotion modelling using neural networks , 2005, Neural Networks.

[76]  Shalabh Bhatnagar,et al.  Natural actor-critic algorithms , 2009, Autom..

[77]  George N. Votsis,et al.  Emotion recognition in human-computer interaction , 2001, IEEE Signal Process. Mag..

[78]  Ingo Siegert,et al.  Towards Emotion and Affect Detection in the Multimodal LAST MINUTE Corpus , 2012, LREC.

[79]  P. Ekman.  Facial expression and emotion. , 1993, The American psychologist.

[80]  Elmar Nöth,et al.  Private emotions versus social interaction: a data-driven approach towards analysing emotion in speech , 2008, User Modeling and User-Adapted Interaction.

[81]  Loïc Kessous,et al.  Multimodal emotion recognition from expressive faces, body gestures and speech , 2007, AIAI.

[82]  Katrin Hille.  Synthesizing Emotional Behavior in a Simple Animated Character , 2001, Artificial Life.

[83]  P. Ekman.  An argument for basic emotions , 1992 .

[84]  Johannes Wagner,et al.  A Systematic Comparison of Different HMM Designs for Emotion Recognition from Acted and Spontaneous Speech , 2007, ACII.

[85]  G. Palm,et al.  Classifier fusion for emotion recognition from speech , 2007 .

[86]  Neil D. Lawrence,et al.  Semi-supervised Learning via Gaussian Processes , 2004, NIPS.

[87]  P. Dayan,et al.  Reward, Motivation, and Reinforcement Learning , 2002, Neuron.

[88]  Vittorio Gallese,et al.  Mirrors of the mind. , 2006, Scientific American.

[89]  J. Bach,et al.  Principles of Synthetic Intelligence: Psi: An Architecture of Motivated Cognition , 2009 .

[90]  Günther Palm,et al.  When classifier selection meets information theory: A unifying view , 2010, 2010 International Conference of Soft Computing and Pattern Recognition.

[91]  Takeo Kanade,et al.  Recognizing Action Units for Facial Expression Analysis , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[92]  Friedhelm Schwenker,et al.  Studying Self- and Active-Training Methods for Multi-feature Set Emotion Recognition , 2011, PSL.

[93]  Friedhelm Schwenker,et al.  Multimodal Emotion Classification in Naturalistic User Behavior , 2011, HCI.

[94]  Nick Campbell,et al.  Multimedia Database of Meetings and Informal Interactions for Tracking Participant Involvement and Discourse Flow , 2006, LREC.

[95]  Günther Palm,et al.  Comparison of Neural Classification Algorithms Applied to Land Cover Mapping , 2009, WIRN.

[96]  Günther Palm,et al.  Information Integration in a Multi-Stage Object Classifier , 2005, AMS.

[97]  M. Tomasello,et al.  Understanding and sharing intentions: The origins of cultural cognition , 2005, Behavioral and Brain Sciences.

[98]  M. Arbib,et al.  Language within our grasp , 1998, Trends in Neurosciences.

[99]  Björn W. Schuller,et al.  Towards More Reality in the Recognition of Emotional Speech , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[100]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[101]  Ingo Siegert,et al.  Appropriate emotional labelling of non-acted speech using basic emotions, geneva emotion wheel and self assessment manikins , 2011, 2011 IEEE International Conference on Multimedia and Expo.

[102]  K. Scherer,et al.  Criteria for Emotion Recognition from Verbal and Nonverbal Expression: Studying Baggage Loss in the Airport , 2000 .

[103]  A R Damasio,et al.  Descartes' error and the future of human life. , 1994, Scientific American.

[104]  A. Goldman,et al.  Mirror neurons and the simulation theory of mind-reading , 1998, Trends in Cognitive Sciences.

[105]  Leonardo Fogassi,et al.  CURRENT DIRECTIONS IN PSYCHOLOGICAL SCIENCE Mirror Neurons and the Evolution of Embodied Language , 2022 .

[106]  Dirk Heylen,et al.  The Sensitive Artificial Listner: an induction technique for generating emotionally coloured conversation , 2008 .

[107]  Loïc Kessous,et al.  Whodunnit - Searching for the most important feature types signalling emotion-related user states in speech , 2011, Comput. Speech Lang..

[108]  J. Allwood.  A Framework for Studying Human Multimodal Communication , 2013 .

[109]  Stacy Marsella,et al.  EMA: A process model of appraisal dynamics , 2009, Cognitive Systems Research.

[110]  K. Scherer.  What are emotions? And how can they be measured? , 2005 .