Facial emotional classification: from a discrete perspective to a continuous emotional space

User emotion detection is a very useful input to develop affective computing strategies in modern human computer interaction. In this paper, an effective system for facial emotional classification is described. The main distinguishing feature of our work is that the system does not simply provide a classification in terms of a set of discrete emotional labels, but that it operates in a continuous 2D emotional space enabling a wide range of intermediary emotional states to be obtained. As output, an expressional face is represented as a point in a 2D space characterized by evaluation and activation factors. The classification method is based on a novel combination of five classifiers and takes into consideration human assessment for the evaluation of the results. The system has been tested with an extensive universal database so that it is capable of analyzing any subject, male or female of any age and ethnicity. The results are very encouraging and show that our classification strategy is consistent with human brain emotional classification mechanisms.

[1]  Ipke Wachsmuth,et al.  Simulating empathy for the virtual human Max , 2007 .

[2]  Ian Witten,et al.  Data Mining , 2000 .

[3]  Frédéric Gosselin,et al.  Bubbles: a technique to reveal the use of information in recognition tasks , 2001, Vision Research.

[4]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2009, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Loïc Kessous,et al.  Modeling naturalistic affective states via facial and vocal expressions recognition , 2006, ICMI '06.

[6]  Thomas S. Huang,et al.  3D facial expression recognition based on automatically selected features , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[7]  Mark A. Hall,et al.  Correlation-based Feature Selection for Machine Learning , 2003 .

[8]  Zhengyou Zhang,et al.  Comparison between geometry-based and Gabor-wavelets-based facial expression recognition using multi-layer perceptron , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[9]  Wenyuan Bi,et al.  Distributing expressional faces in 2-D emotional space , 2007, CIVR '07.

[10]  Hongbin Zha,et al.  Modeling facial expression space for recognition , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[11]  R. Plutchik.  Emotion, a psychoevolutionary synthesis , 1980 .

[12]  Mohammed Yeasin,et al.  Recognition of facial expressions and measurement of levels of interest from video , 2006, IEEE Transactions on Multimedia.

[13]  Gwen Littlewort,et al.  Dynamics of Facial Expression Extracted Automatically from Video , 2004, CVPR Workshops.

[14]  Young-suk Shin.  Facial Expression Recognition Based on Emotion Dimensions on Manifold Learning , 2007, International Conference on Computational Science.

[15]  M. Bartlett,et al.  Machine Analysis of Facial Expressions , 2007 .

[16]  T. Dalgleish,et al.  Handbook of cognition and emotion , 1999 .

[17]  Qiang Ji,et al.  Active and dynamic information fusion for facial expression understanding from image sequences , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Björn W. Schuller,et al.  Emotion representation, analysis and synthesis in continuous space: A survey , 2011, Face and Gesture 2011.

[19]  Piyush Kumar,et al.  Minimum-Volume Enclosing Ellipsoids and Core Sets , 2005 .

[20]  Ioannis Pitas,et al.  Facial Expression Recognition in Image Sequences Using Geometric Deformation Features and Support Vector Machines , 2007, IEEE Transactions on Image Processing.

[21]  Jennifer S. Beer,et al.  Facial expression of emotion. , 2003 .

[22]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[23]  อนิรุธ สืบสิงห์,et al.  Data Mining Practical Machine Learning Tools and Techniques , 2014 .

[24]  Ashish Kapoor,et al.  Automatic prediction of frustration , 2007, Int. J. Hum. Comput. Stud..

[25]  Gwen Littlewort,et al.  Analysis of Machine Learning Methods for Real-Time Recognition of Facial Expressions from Video , 2003 .

[26]  J. G. Taylor,et al.  Emotion recognition in human-computer interaction , 2005, Neural Networks.

[27]  Cynthia Whissell,et al.  THE DICTIONARY OF AFFECT IN LANGUAGE , 1989 .

[28]  J. N. Bassili.  Emotion recognition: the role of facial movement and the relative importance of upper and lower areas of the face. , 1979, Journal of personality and social psychology.

[29]  Zakia Hammal,et al.  Facial expression classification: An approach based on the fusion of facial deformations using the transferable belief model , 2007, Int. J. Approx. Reason..

[30]  Maja Pantic,et al.  Web-based database for facial expression analysis , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[31]  Mann Oo. Hay.  Emotion recognition in human-computer interaction , 2012 .

[32]  Barbara L. Fredrickson,et al.  The Value of Positive Emotions , 2003, American Scientist.

[33]  Hua-jun Song,et al.  An Expression Space Model for Facial Expression Analysis , 2008, 2008 Congress on Image and Signal Processing.

[34]  Renaud Séguier,et al.  Automatic design of a control interface for a synthetic face , 2009, IUI.

[35]  Hasan Demirel,et al.  Facial Expression Recognition Using 3D Facial Feature Distances , 2007, ICIAR.

[36]  Qiang Ji,et al.  A probabilistic framework for modeling and real-time monitoring human fatigue , 2006, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[37]  Chuan-Yu Chang,et al.  Emotion recognition with consideration of facial expression and physiological signals , 2009, 2009 IEEE Symposium on Computational Intelligence in Bioinformatics and Computational Biology.

[38]  Nicu Sebe,et al.  Facial expression recognition from video sequences: temporal and static modeling , 2003, Comput. Vis. Image Underst..

[39]  Peter W. McOwan,et al.  A real-time automated system for the recognition of human facial expressions , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[40]  Léon J. M. Rothkrantz,et al.  Facial expression recognition in still pictures and videos using active appearance models: a comparison approach , 2007, CompSysTech '07.

[41]  Simon Lucey,et al.  Investigating Spontaneous Facial Action Recognition through AAM Representations of the Face , 2007 .

[42]  Stefano Berretti,et al.  Shape analysis of local facial patches for 3D facial expression recognition , 2011, Pattern Recognit..