Hybrid feature extraction for facial emotion recognition

In recent advances of technology, the application of facial emotion recognition for human-computer interaction (HCI) is becoming an emerging trend. This HCI depends to a large extent on its ability to recognise the facial expression and ability to withstand various kinds of noise. However, confidence in its ability in providing adequate recognition remains challenging due to subtlety, complexity, and variability of facial expression. Therefore, this paper proposes hybrid feature extraction using 2D discrete wavelet transform (DWT), principal component analysis (PCA), and linear discriminant analysis (LDA) for seven (angry, disgust, fear, happy, neutral, sad, and surprise) different facial emotion recognition from static images. The reduced features are tested using k-nearest-neighbour classifier to recognise the facial emotion. The proposed method is evaluated based on two different databases, namely Japanese female facial expression and Cohn-Kanade database. The proposed method gives promising recognition rates of 100% for JAFFE and 97.52% for Cohn-Kanade. The effect of different wavelet families on the classification performance is also investigated.

[1]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Larry S. Davis,et al.  Human expression recognition from motion using a radial basis function network architecture , 1996, IEEE Trans. Neural Networks.

[3]  Larry S. Davis,et al.  Computing spatio-temporal representations of human faces , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Sazali Yaacob,et al.  A comparative study of wavelet families for classification of wrist motions , 2012, Comput. Electr. Eng..

[5]  M. Arfan Jaffar,et al.  Wavelets-based facial expression recognition using a bank of support vector machines , 2012, Soft Comput..

[6]  Y. V. Venkatesh,et al.  Facial expression recognition using radial encoding of local Gabor features and classifier synthesis , 2012, Pattern Recognit..

[7]  Shiqing Zhang,et al.  Facial Expression Recognition Based on Local Binary Patterns and Kernel Discriminant Isomap , 2011, Sensors.

[8]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[9]  Sazali Yaacob,et al.  Infant cry classification to identify asphyxia using time-frequency analysis and radial basis neural networks , 2012, Expert Syst. Appl..

[10]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[11]  Beat Fasel,et al.  Automatic facial expression analysis: a survey , 2003, Pattern Recognit..

[12]  Matti Pietikäinen,et al.  A Coarse-to-Fine Classification Scheme for Facial Expression Recognition , 2004, ICIAR.

[13]  J. Ohya,et al.  Recognition of facial expressions using HMM with continuous output probabilities , 1996, Proceedings 5th IEEE International Workshop on Robot and Human Communication. RO-MAN'96 TSUKUBA.

[14]  Larry S. Davis,et al.  Recognizing Human Facial Expressions From Long Image Sequences Using Optical Flow , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  M. M. Mohie El-din,et al.  Hybrid Framework for Robust Multimodal Face Recognition , 2013 .

[16]  Michael J. Lyons,et al.  Automatic Classification of Single Facial Images , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Jun Ou,et al.  Automatic Facial Expression Recognition Using Gabor Filter and Expression Analysis , 2010, 2010 Second International Conference on Computer Modeling and Simulation.

[18]  Frank Y. Shih,et al.  Automatic extraction of head and face boundaries and facial features , 2004, Inf. Sci..

[19]  Jen-Tzung Chien,et al.  Discriminant Waveletfaces and Nearest Feature Classifiers for Face Recognition , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Engin Avci,et al.  An expert system based on Wavelet Neural Network-Adaptive Norm Entropy for scale invariant texture classification , 2007, Expert Syst. Appl..

[21]  Vytautas Perlibakas.  Face Recognition Using Principal Component Analysis and Wavelet Packet Decomposition , 2004, Informatica.

[22]  Alex Pentland,et al.  Face recognition using eigenfaces , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  A. Mehrabian.  Communication without words , 1968 .

[24]  Marian Stewart Bartlett,et al.  Classifying Facial Action , 1995, NIPS.

[25]  Y. V. Venkatesh,et al.  A novel application of self-organizing network for facial expression recognition from radial encoded contours , 2009, Soft Comput..

[26]  Dian Tjondronegoro,et al.  Facial Expression Recognition Using Facial Movement Features , 2011, IEEE Transactions on Affective Computing.

[27]  Kenji Mase,et al.  Recognition of Facial Expression from Optical Flow , 1991 .

[28]  Sazali Yaacob,et al.  Classification of speech dysfluencies with MFCC and LPCC features , 2012, Expert Syst. Appl..

[29]  Alex Pentland,et al.  Coding, Analysis, Interpretation, and Recognition of Facial Expressions , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  M. Hariharan,et al.  MFCC based recognition of repetitions and prolongations in stuttered speech using k-NN and LDA , 2009, 2009 IEEE Student Conference on Research and Development (SCOReD).

[31]  Lawrence Sirovich,et al.  Application of the Karhunen-Loeve Procedure for the Characterization of Human Faces , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Avinash C. Kak,et al.  PCA versus LDA , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[34]  Patrick Shen-Pei Wang,et al.  Performance Comparisons of Facial Expression Recognition in Jaffe Database , 2008, Int. J. Pattern Recognit. Artif. Intell..

[35]  M. Hariharan,et al.  Identification of vocal fold pathology based on Mel Frequency Band Energy Coefficients and singular value decomposition , 2009, 2009 IEEE International Conference on Signal and Image Processing Applications.

[36]  Marian Stewart Bartlett,et al.  Classifying Facial Actions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..