Frontalization with Adaptive Exponentially-Weighted Average Ensemble Rule for Deep Learning Based Facial Expression Recognition

Automatic Facial Expression Recognition (FER) is an important technique in human-computer interfaces and surveillance systems. It classifies the input facial images into one of the basic expressions (e.g., anger, sad, surprise, happy, disgust, fear, and neutral) and has attracted significant attention in pattern recognition and computer vision. In this paper, we proposed an advanced convolutional neural networks based FER system. It applies the techniques of face frontalization and feature positioning to reduce the effects of background noise and non-prominent parts. Moreover, the hierarchical structure together with the adaptive exponentially weighted average ensemble are adopted to further improve the accuracies. Simulations on several datasets show that the proposed system outperform state-of-the-art FER methods and can well identify the expression of a person.

[1]  Ping Liu,et al.  Facial Expression Recognition via a Boosted Deep Belief Network , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Junmo Kim,et al.  Joint Fine-Tuning in Deep Neural Networks for Facial Expression Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[3]  Radu Tudor Ionescu,et al.  Local Learning to Improve Bag of Visual Words Model for Facial Expression Recognition , 2013 .

[4]  Christine L. Lisetti,et al.  Facial Expression Recognition Using a Neural Network , 1998, FLAIRS.

[5]  Yichuan Tang,et al.  Deep Learning using Linear Support Vector Machines , 2013, 1306.0239.

[6]  Shiguang Shan,et al.  Deeply Learning Deformable Facial Action Parts Model for Dynamic Expression Analysis , 2014, ACCV.

[7]  Guodong Guo,et al.  Simultaneous feature selection and classifier training via linear programming: a case study for face expression recognition , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[8]  Graham W. Taylor,et al.  Multi-task Learning of Facial Landmarks and Expression , 2014, 2014 Canadian Conference on Computer and Robot Vision.

[9]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[10]  P. Ekman.  Facial expression and emotion. , 1993, The American psychologist.

[11]  Ah Chung Tsoi,et al.  Face recognition: a convolutional neural-network approach , 1997, IEEE Trans. Neural Networks.

[12]  Soo-Young Lee,et al.  Hierarchical committee of deep convolutional neural networks for robust facial expression recognition , 2016, Journal on Multimodal User Interfaces.

[13]  Aurobinda Routray,et al.  Automatic facial expression recognition using features of salient facial patches , 2015, IEEE Transactions on Affective Computing.

[14]  Cüneyt Güzelis,et al.  A new facial expression recognition based on curvelet transform and online sequential extreme learning machine initialized with spherical clustering , 2014, Neural Computing and Applications.

[15]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[16]  Edilson de Aguiar,et al.  A Facial Expression Recognition System Using Convolutional Networks , 2015, 2015 28th SIBGRAPI Conference on Graphics, Patterns and Images.

[17]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Shen-Chuan Tai,et al.  Facial Expression Recognition using Neural Network , 2007 .

[19]  Jian-Jiun Ding,et al.  Facial expression recognition based on improved local binary pattern and class-regularized locality preserving projection , 2015, Signal Process..

[20]  M. Pandya,et al.  A Survey of Face Recognition approach , 2013 .