Effective semantic features for facial expressions recognition using SVM

Most traditional facial expression-recognition systems track facial components such as eyes, eyebrows, and mouth for feature extraction. Though some of these features can provide clues for expression recognition, other finer changes of the facial muscles can also be deployed for classifying various facial expressions. This study locates facial components by active shape model to extract seven dynamic face regions (frown, nose wrinkle, two nasolabial folds, two eyebrows, and mouth). Proposed semantic facial features could then be acquired using directional gradient operators like Gabor filters and Laplacian of Gaussian. A multi-class support vector machine (SVM) was trained to classify six facial expressions (neutral, happiness, surprise, anger, disgust, and fear). The popular Cohn–Kanade database was tested and the average recognition rate reached 94.7 %. Also, 20 persons were invited for on-line test and the recognition rate was about 93 % in a real-world environment. It demonstrated that the proposed semantic facial features could effectively represent changes between facial expressions. The time complexity could be lower than the other SVM based approaches due to the less number of deployed features.

[1]  Franck Davoine,et al.  Facial expression recognition and synthesis based on an appearance model , 2004, Signal Process. Image Commun..

[2]  Bo Wu,et al.  Real time facial expression recognition with AdaBoost , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[3]  Ayoub Al-Hamadi,et al.  Frame-Based Facial Expression Recognition Using Geometrical Features , 2014, Adv. Hum. Comput. Interact..

[4]  Beat Fasel,et al.  Automati Fa ial Expression Analysis: A Survey , 1999 .

[5]  P SumathiC.,et al.  Automatic Facial Expression Analysis A Survey , 2012 .

[6]  Ching-Nung Yang,et al.  Advances in Intelligent Systems and Applications - Volume 2 , 2013 .

[7]  Timothy F. Cootes,et al.  Automatic Interpretation and Coding of Face Images Using Flexible Models , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Matti Pietikäinen,et al.  Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Salvatore Tabbone.  Detecting junctions using properties of the Laplacian of Gaussian detector , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[10]  Erik G. Learned-Miller,et al.  Online domain adaptation of a pre-trained cascade of classifiers , 2011, CVPR 2011.

[11]  Yantao Tian,et al.  Rapid Face Detection Algorithm of Color Images under Complex Background , 2011, ISNN.

[12]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[13]  Deva Ramanan,et al.  Face detection, pose estimation, and landmark localization in the wild , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[15]  Gihan Shin,et al.  Spatio-temporal Facial Expression Recognition Using Optical Flow and HMM , 2008, Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing.

[16]  Rainer Lienhart,et al.  An extended set of Haar-like features for rapid object detection , 2002, Proceedings. International Conference on Image Processing.

[17]  Anthony J. T. Lee,et al.  A data mining approach to face detection , 2010, Pattern Recognit..

[18]  Marian Stewart Bartlett,et al.  Facial expression recognition using Gabor motion energy filters , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[19]  Ganesh K. Venayagamoorthy,et al.  Recognition of facial expressions using Gabor wavelets and learning vector quantization , 2008, Eng. Appl. Artif. Intell..

[20]  Vinay Bettadapura,et al.  Face Expression Recognition and Analysis: The State of the Art , 2012, ArXiv.

[21]  Benzai Deng,et al.  Facial Expression Recognition using AAM and Local Facial Features , 2007, Third International Conference on Natural Computation (ICNC 2007).

[22]  Federica Marcolin,et al.  3D Landmarking in Multiexpression Face Analysis: A Preliminary Study on Eyebrows and Mouth , 2014, Aesthetic Plastic Surgery.

[23]  Hakan Cevikalp,et al.  Face and landmark detection by using cascade of classifiers , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[24]  Maja Pantic,et al.  Fully automatic facial feature point detection using Gabor feature based boosted classifiers , 2005, 2005 IEEE International Conference on Systems, Man and Cybernetics.

[25]  Matti Pietikäinen,et al.  Boosted multi-resolution spatiotemporal descriptors for facial expression recognition , 2009, Pattern Recognit. Lett..

[26]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[27]  Gwen Littlewort,et al.  Fully Automatic Facial Action Recognition in Spontaneous Behavior , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[28]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[29]  N. Tsapatsoulis,et al.  Comparing Template-based , Feature-based and Supervised Classification of Facial Expressions from Static Images , 1999 .

[30]  Lijun Yin,et al.  Static and dynamic 3D facial expression recognition: A comprehensive survey , 2012, Image Vis. Comput..

[31]  Halim Fathoni,et al.  DEPARTMENT OF COMPUTER SCIENCE AND INFORMATION ENGINEERING , 2008 .

[32]  S. Sathiya Keerthi,et al.  Which Is the Best Multiclass SVM Method? An Empirical Study , 2005, Multiple Classifier Systems.

[33]  Chih-Jen Lin,et al.  A Practical Guide to Support Vector Classication , 2008 .

[34]  Zhang Youwei,et al.  Facial expression recognition based on two dimensional feature extraction , 2008, 2008 9th International Conference on Signal Processing.

[35]  Rui Ma,et al.  Automatic Facial Expression Recognition Using Linear and Nonlinear Holistic Spatial Analysis , 2005, ACII.

[36]  Alison Griffiths,et al.  Multiple face detection algorithm using colour skin modelling , 2012 .

[37]  Friedhelm Schwenker,et al.  A Hidden Markov Model Based Approach for Facial Expression Recognition in Image Sequences , 2010, ANNPR.

[38]  Yi-Cheng Zhang,et al.  Using SVM to design facial expression recognition for shape and texture features , 2010, 2010 International Conference on Machine Learning and Cybernetics.

[39]  Wilhelm Burger,et al.  Digital Image Processing - An Algorithmic Introduction using Java , 2008, Texts in Computer Science.

[40]  Josef Kittler,et al.  Multiple Classifier Systems , 2004, Lecture Notes in Computer Science.

[41]  Kun Zhang,et al.  Facial expression recognition using geometric and appearance features , 2012, ICIMCS '12.

[42]  Jun Ou,et al.  Automatic Facial Expression Recognition Using Gabor Filter and Expression Analysis , 2010, 2010 Second International Conference on Computer Modeling and Simulation.

[43]  Michal Kawulok,et al.  Precise multi-level face detector for advanced analysis of facial images , 2012 .

[44]  Chung-Lin Huang,et al.  Facial Expression Recognition Using Model-Based Feature Extraction and Action Parameters Classification , 1997, J. Vis. Commun. Image Represent..

[45]  F. Hara,et al.  Facial interaction between animated 3D face robot and human beings , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[46]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[47]  Jon Atli Benediktsson,et al.  Multiple Classifier Systems , 2015, Lecture Notes in Computer Science.

[48]  Richard Bowden,et al.  Local binary patterns for multi-view facial expression recognition , 2011 .

[49]  Andrew J. Calder,et al.  PII: S0042-6989(01)00002-5 , 2001 .

[50]  Oscar Déniz-Suárez,et al.  A comparison of face and facial feature detectors based on the Viola–Jones general object detection framework , 2011, Machine Vision and Applications.

[51]  Hsi-Chieh Lee,et al.  Facial Expression Recognition Using Image Processing Techniques and Neural Networks , 2013 .

[52]  J. Daugman.  Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[53]  Federica Marcolin,et al.  3D human face description: landmarks measures and geometrical features , 2012, Image Vis. Comput..

[54]  Shane Xie,et al.  Automated facial expression recognition - an integrated approach with optical flow analysis and Support Vector Machines , 2009, Int. J. Intell. Syst. Technol. Appl..

[55]  Maja Pantic,et al.  Fully Automatic Recognition of the Temporal Phases of Facial Actions , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[56]  Ting Wu,et al.  Survey of the Facial Expression Recognition Research , 2012, BICS.