A Computational Intelligence Approach to Emotion Recognition from the Lip-Contour of a Subject

This chapter provides an alternative approach to emotion recognition from the outer lip-contour of the subjects. Subjects exhibit their emotions through their facial expressions, and the lip region is segmented from their facial images. A lip-contour model has been developed to represent the boundary of the lip, and the parameters of the model are adapted using differential evolution algorithm to match it with the boundary contour of the lip. A support vector machine (SVM) classifier is then employed to classify the emotion of the subject from the parameter set of the subjects’ lip-contour. The experiment was performed on 50 subjects in an age group from 18 to 25, and the average case accuracy in emotion classification is found to be 86 %.

[1]  Fumio Hara,et al.  Recognition of Mixed Facial Expressions by Neural Network. , 1993 .

[2]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[3]  Xiangdong Wang,et al.  Accurate and real-time lip contour extraction based on constrained contour growing , 2009, 2009 Joint Conferences on Pervasive Computing (JCPC).

[4]  Maja Pantic,et al.  Dynamics of facial expression: recognition of facial actions and their temporal segments from face profile image sequences , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[5]  Nicu Sebe,et al.  Facial expression recognition from video sequences , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[6]  Hans-Jürgen Zimmermann,et al.  Fuzzy Set Theory - and Its Applications , 1985 .

[7]  Carlos Busso,et al.  Interrelation Between Speech and Facial Gestures in Emotional Utterances: A Single Subject Study , 2007, IEEE Transactions on Audio, Speech, and Language Processing.

[8]  Timothy F. Cootes,et al.  Automatic Interpretation and Coding of Face Images Using Flexible Models , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  S. Yaacob,et al.  Japanese Face Emotions Classification Using LIP Features , 2007, Geometric Modeling and Imaging (GMAI '07).

[10]  Amit Konar,et al.  A support vector machine classifier of emotion from voice and facial expression data , 2009, 2009 World Congress on Nature & Biologically Inspired Computing (NaBIC).

[11]  Amit Konar,et al.  Computational Intelligence: Principles, Techniques and Applications , 2005 .

[12]  Yuxiao Hu,et al.  Spontaneous Emotional Facial Expression Detection , 2006, J. Multim..

[13]  Fumio Hara,et al.  The recognition of basic facial expressions by neural network , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[14]  Nicu Sebe,et al.  Facial expression recognition from video sequences: temporal and static modeling , 2003, Comput. Vis. Image Underst..

[15]  Lionel Revéret,et al.  A New 3D Lip Model for Analysis and Synthesis of Lip Motion In Speech Production , 1998, AVSP.

[16]  Xavier Maldague,et al.  Unsupervised Lips Segmentation Based on ROI Optimisation and Parametric Model , 2007, 2007 IEEE International Conference on Image Processing.

[17]  T. Saitoh,et al.  Analysis of Efficient Feature for Japanese Vowel Recognition , 2006, 2006 International Symposium on Intelligent Signal Processing and Communications.

[18]  Lipo Wang,et al.  Support Vector Machines: Theory and Applications (Studies in Fuzziness and Soft Computing) , 2005 .

[19]  S. Yaacob,et al.  Fuzzy clustering for genetic algorithm based optimized ellipse data in classifying face emotion , 2007, 2007 International Conference on Control, Automation and Systems.

[20]  P. Ekman,et al.  Unmasking the face : a guide to recognizing emotions from facial clues , 1975 .

[21]  Liang Yaling,et al.  Lip Contour Extraction Based on Manifold , 2008, 2008 International Conference on MultiMedia and Information Technology.

[22]  A. Konar,et al.  Voice and Facial Expression Based Classification of Emotion Using Linear Support Vector Machine , 2009, 2009 Second International Conference on Developments in eSystems Engineering.

[23]  Ganesh K. Venayagamoorthy,et al.  Recognition of facial expressions using Gabor wavelets and learning vector quantization , 2008, Eng. Appl. Artif. Intell..

[24]  Amit Konar,et al.  A recurrent neural model for parameter estimation of mixed emotions from facial expressions of the subjects , 2009, 2009 International Joint Conference on Neural Networks.

[25]  H. Yamada.  Visual Information for Categorizing Facial Expression of Emotions , 1993 .

[26]  Amit Konar,et al.  Emotional Intelligence: A Cybernetic Approach , 2009, Studies in Computational Intelligence.

[27]  Siu Cheung Hui,et al.  Facial expression recognition from line-based caricatures , 2003, IEEE Trans. Syst. Man Cybern. Part A.

[28]  Amit Konar,et al.  Emotion Recognition From Facial Expressions and Its Control Using Fuzzy Logic , 2009, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[29]  Fumio Hara,et al.  Measurement of the Strength of Six Basic Facial Expressions by Neural Network. , 1993 .

[30]  H. Wallbott,et al.  Emotion category accessibility and the decoding of emotion from facial expression and context , 1991 .