Frontal Facial Expression Recognition using Parallel CNN Model

Facial expression recognition is one of the very important research topics in computer vision. Studies on nonverbal communication have shown that 55% of intentional information is conveyed through facial expressions. Expression recognition has recently found a lot many applications in medical and advertising industries. In this paper we have proposed a parallel Convolutional Neural Network (CNN) structure for detection of expression from frontal faces. The CNNs are trained on two most important subfacial patches. The overall feature vector will be the features concatenated from the parallel models. We have experimentally found applying such a strategy provides better results than the models which take the entire facial image. We have also compared our performance with other benchmark CNN structures like AlexNet and VGG16.

[1]  P. Suja,et al.  Emotion recognition from 3D videos using optical flow method , 2017, 2017 International Conference On Smart Technologies For Smart Nation (SmartTechCon).

[2]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[3]  Aurobinda Routray,et al.  A real time facial expression classification system using Local Binary Patterns , 2015, 2012 4th International Conference on Intelligent Human Computer Interaction (IHCI).

[4]  Haifeng Hu,et al.  Facial Expression Recognition Using Hierarchical Features With Deep Comprehensive Multipatches Aggregation Convolutional Neural Networks , 2019, IEEE Transactions on Multimedia.

[5]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[6]  Dong-Soo Kwon,et al.  Baseline CNN structure analysis for facial expression recognition , 2016, 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[7]  Jun Ou,et al.  Automatic Facial Expression Recognition Using Gabor Filter and Expression Analysis , 2010, 2010 Second International Conference on Computer Modeling and Simulation.

[8]  Georgios Tzimiropoulos,et al.  How Far are We from Solving the 2D & 3D Face Alignment Problem? (and a Dataset of 230,000 3D Facial Landmarks) , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[9]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[10]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[11]  Junmo Kim,et al.  Joint Fine-Tuning in Deep Neural Networks for Facial Expression Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[12]  Anima Majumder,et al.  Automatic Facial Expression Recognition System Using Deep Network-Based Data Fusion , 2018, IEEE Transactions on Cybernetics.

[13]  Oscar Déniz-Suárez,et al.  Face recognition using Histograms of Oriented Gradients , 2011, Pattern Recognit. Lett..

[14]  Joachim M. Buhmann,et al.  TI-POOLING: Transformation-Invariant Pooling for Feature Learning in Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Yibin Li,et al.  Facial Expression Recognition with Fusion Features Extracted from Salient Facial Areas , 2017, Sensors.

[16]  Takeo Kanade,et al.  Multi-PIE , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[17]  Yichuan Tang,et al.  Deep Learning using Support Vector Machines , 2013, ArXiv.

[18]  Cha Zhang,et al.  Image based Static Facial Expression Recognition with Multiple Deep Network Learning , 2015, ICMI.

[19]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[20]  P. Ekman,et al.  Facial Action Coding System: Manual , 1978 .

[21]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[22]  Maja Pantic,et al.  Action unit detection using sparse appearance descriptors in space-time video volumes , 2011, Face and Gesture 2011.

[23]  Razvan Pascanu,et al.  Combining modality specific deep neural networks for emotion recognition in video , 2013, ICMI '13.

[24]  Deepak Ghimire,et al.  Geometric Feature-Based Facial Expression Recognition in Image Sequences Using Multi-Class AdaBoost and Support Vector Machines , 2013, Sensors.