Convolutional Neural Network Based Expression Classification with Face Alignment

Convolutional neural networks (CNNs)have been proved to be excellent in image classification including facial expression recognition. However, CNNs commonly fail to learn the key expression characteristics with different facial angles. In this paper, the facial expression recognition problem is considered. The facial alignment technology is employed to adjust the inclined faces where an affine transformation is utilized such that the faces lie on a horizontal line. Then, in consideration of the scale of the model parameters, a sequential CNN is designed to extract the facial features, where a global averaged pooling layer is adopted instead of the fully connected layer. Comparing with the data augmentation, the experimental results on the three benchmark datasets (ie., Jaffe, CK+ and FER-2013)show that better performance is guaranteed by our model with less adjustable parameters.

[1]  Nikolay Neshov,et al.  Expression Recognition Using Sparse Selection of log-Gabor Facial Features , 2017, 2017 Fourth International Conference on Mathematics and Computers in Sciences and in Industry (MCSI).

[2]  Dinh Viet Sang,et al.  Facial expression recognition using deep convolutional neural networks , 2017, 2017 9th International Conference on Knowledge and Systems Engineering (KSE).

[3]  Li Zhuo,et al.  Learning realistic facial expressions from web images , 2013, Pattern Recognit..

[4]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[5]  Anja Dieckmann,et al.  Towards Robust Real-Time Valence Recognition from Facial Expressions for Market Research Applications , 2013, 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction.

[6]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[7]  Huiquan Zhang,et al.  Facial Expressions Recognition Based on Cognition and Mapped Binary Patterns , 2018, IEEE Access.

[8]  Jiang Yi-lin.  Performance analysis of image preprocessing technology based on background estimation , 2004 .

[9]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[10]  Jean Ponce,et al.  Learning mid-level features for recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Arash Habibi Lashkari,et al.  FACIAL EXPRESSION RECOGNITION INTELLIGENT SECURITY SYSTEM FOR REAL TIME SURVEILLANCE , 2012 .

[12]  Michael J. Lyons,et al.  Automatic Classification of Single Facial Images , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Lin Ma,et al.  Multimodal learning for facial expression recognition , 2015, Pattern Recognit..

[14]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Andrew Zisserman,et al.  Spatial Transformer Networks , 2015, NIPS.

[16]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[17]  M. Munafo,et al.  Meta-analysis of emotion recognition deficits in major depressive disorder , 2014, Psychological Medicine.

[18]  Sepp Hochreiter,et al.  Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) , 2015, ICLR.

[19]  Yan Liu,et al.  A new method of feature fusion and its application in image recognition , 2005, Pattern Recognit..

[20]  Gyanendra K. Verma.  Facial micro-expression recognition using discrete curvelet transform , 2017, 2017 Conference on Information and Communication Technology (CICT).

[21]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[22]  Daniel Lélis Baggio,et al.  Mastering OpenCV with Practical Computer Vision Projects , 2012 .

[23]  Wanping Li,et al.  Emotion Analysis Based on Facial Expression Recognition in Virtual Learning Environment , 2017 .

[24]  Haifeng Hu,et al.  Facial expression recognition with FRR-CNN , 2017 .

[25]  Yoshua Bengio,et al.  Challenges in representation learning: A report on three machine learning contests , 2013, Neural Networks.

[26]  Eugenio Culurciello,et al.  An Analysis of Deep Neural Network Models for Practical Applications , 2016, ArXiv.

[27]  Pasquale Pagano,et al.  OpenDLib: A Digital Library Service System , 2002, ECDL.

[28]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[29]  Qiang Chen,et al.  Network In Network , 2013, ICLR.