Using multiple steerable filters and Bayesian regularization for facial expression recognition

Abstract Facial expression recognition has recently become a challenging research area. Its applications include human–computer interfaces, human emotion analysis, and medical care and cure. In this paper, we present a new challenging method to recognize seven universal emotional expressions, which are happiness, neutral, angry, disgust, sadness, fear and surprise. In our approach, we identify the user׳s facial expressions from the input images, using statistical features extracted from the steerable pyramid decomposition, and classified with a Bayesian regularization neural network. The evaluation of the proposed approach in terms of recognition accuracy is achieved using two universal databases, the Japanese Female Facial Expression database and the Cohn–Kanade facial expression database. The overall accuracy rate reaches 93.33% for the first database and is about 98.13% for the second one. These results show the effectiveness of the steerable proposed algorithm.

[1]  Raúl Rojas,et al.  Neural Networks - A Systematic Introduction , 1996 .

[2]  Yegui Xiao,et al.  A New Facial Expression Recognition Technique using 2-D DCT and Neural Networks Based Decision Tree , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[3]  M. Osman Tokhi,et al.  A fuzzy clustering neural network architecture for multifunction upper-limb prosthesis , 2003, IEEE Transactions on Biomedical Engineering.

[4]  M. Pietikäinen,et al.  Facial expression recognition based on local binary patterns , 2007, Pattern Recognition and Image Analysis.

[5]  Alberto Del Bimbo,et al.  3D facial expression recognition using SIFT descriptors of automatically detected keypoints , 2011, The Visual Computer.

[6]  David J. C. MacKay,et al.  A Practical Bayesian Framework for Backpropagation Networks , 1992, Neural Computation.

[7]  Beat Fasel,et al.  Automati Fa ial Expression Analysis: A Survey , 1999 .

[8]  Weifeng Liu,et al.  Facial Expression Recognition Based on Fusion of Multiple Gabor Features , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[9]  M. Arfan Jaffar,et al.  Wavelets Based Facial Expression Recognition Using a Bank of Neural Networks , 2010, 2010 5th International Conference on Future Information Technology.

[10]  Edward H. Adelson,et al.  The Design and Use of Steerable Filters , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Franck Davoine,et al.  A solution for facial expression representation and recognition , 2002, Signal Process. Image Commun..

[12]  A. Mehrabian.  Communication without words , 1968 .

[13]  Michel Valstar,et al.  Automatic Facial Expression Analysis , 2015 .

[14]  William T. Freeman,et al.  Presented at: 2nd Annual IEEE International Conference on Image , 1995 .

[15]  Siu Cheung Hui,et al.  Facial expression recognition from line-based caricatures , 2003, IEEE Trans. Syst. Man Cybern. Part A.

[16]  Patrick Shen-Pei Wang,et al.  Performance Comparisons of Facial Expression Recognition in Jaffe Database , 2008, Int. J. Pattern Recognit. Artif. Intell..

[17]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[18]  Martin T. Hagan,et al.  Gauss-Newton approximation to Bayesian learning , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[19]  DornaikaFadi,et al.  Inferring facial expressions from videos , 2007 .

[20]  Daijin Kim,et al.  Natural facial expression recognition using differential-AAM and manifold learning , 2009, Pattern Recognit..

[21]  Bogdan Raducanu,et al.  Inferring facial expressions from videos: Tool and application , 2007, Signal Process. Image Commun..

[22]  Michael J. Lyons,et al.  Coding facial expressions with Gabor wavelets , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[23]  Edward H. Adelson,et al.  Shiftable multiscale transforms , 1992, IEEE Trans. Inf. Theory.

[24]  Eero P. Simoncelli,et al.  A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coefficients , 2000, International Journal of Computer Vision.

[25]  Ioannis Pitas,et al.  ICA and Gabor representation for facial expression recognition , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[26]  S.C. Tai,et al.  Automatic facial expression recognition system using Neural Networks , 2007, TENCON 2007 - 2007 IEEE Region 10 Conference.

[27]  Nobuyuki Otsu,et al.  Facial expression recognition using Fisher weight maps , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[28]  Maja Pantic,et al.  Coupled Gaussian processes for pose-invariant facial expression recognition , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Ioannis Pitas,et al.  Facial Expression Recognition in Image Sequences Using Geometric Deformation Features and Support Vector Machines , 2007, IEEE Transactions on Image Processing.

[30]  Franck Davoine,et al.  Facial expression recognition and synthesis based on an appearance model , 2004, Signal Process. Image Commun..

[31]  PanticM.,et al.  Dynamics of facial expression , 2006 .

[32]  Maja Pantic,et al.  Dynamics of facial expression: recognition of facial actions and their temporal segments from face profile image sequences , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[33]  Matti Pietikäinen,et al.  Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Maja Pantic,et al.  Expert system for automatic analysis of facial expressions , 2000, Image Vis. Comput..

[35]  Ashraf A. Kassim,et al.  A novel approach to classification of facial expressions from 3D-mesh datasets using modified PCA , 2009, Pattern Recognit. Lett..

[36]  Neucimar J. Leite,et al.  Rotation-Invariant and Scale-Invariant Steerable Pyramid Decomposition for Texture Image Retrieval , 2007 .

[37]  Suphakant Phimoltares,et al.  Facial expression recognition using graph-based features and artificial neural networks , 2011, 2011 IEEE International Conference on Imaging Systems and Techniques.

[38]  Zhengyou Zhang,et al.  Comparison between geometry-based and Gabor-wavelets-based facial expression recognition using multi-layer perceptron , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[39]  P. Ekman,et al.  DIFFERENCES Universals and Cultural Differences in the Judgments of Facial Expressions of Emotion , 2004 .

[40]  Michael J. Lyons,et al.  Automatic Classification of Single Facial Images , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[41]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..