Facial emotion detection using modified eyemap–mouthmap algorithm on an enhanced image and classification with tensorflow

Detection of emotion using facial expression is a growing field of research. Facial expression detection is also helpful to identify the behavior of a person when a man interacts with the computer. In this work, facial expression recognition with respect to the changes in the facial geometry is proposed. First, the image is enhanced by means of discrete wavelet transform and fuzzy combination. Then, the facial geometry is found using the modified eyemap and mouthmap algorithm after finding the landmarks. Finally, the area and angle of the constructed triangles are found and classified using neural network with the help of tensorflow central processing unit version. Results show that the proposed algorithm is efficient in finding the facial emotion.

[1]  Dipti Prasad Mukherjee,et al.  Anubhav: recognizing emotions through facial expression , 2016, The Visual Computer.

[2]  Jörgen Ahlberg,et al.  Fast facial expression recognition using local binary features and shallow neural networks , 2018, The Visual Computer.

[3]  Yassine Ruichek,et al.  Facial decomposition for expression recognition using texture/shape descriptors and SVM classifier , 2017, Signal Process. Image Commun..

[4]  N. Alugupally,et al.  Analysis of landmarks in recognition of face expressions , 2011, Pattern Recognition and Image Analysis.

[5]  Aurobinda Routray,et al.  Automatic facial expression recognition using features of salient facial patches , 2015, IEEE Transactions on Affective Computing.

[6]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[7]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[8]  Kranthi A. Lammatha,et al.  Data Mining Object Match Swapping Detection of Facial Landmarks Using Local-based Information , 2019 .

[9]  Subrat Prasad Panda.  Image contrast enhancement in spatial domain using fuzzy logic based interpolation method , 2016, 2016 IEEE Students' Conference on Electrical, Electronics and Computer Science (SCEECS).

[10]  Hamed Shah-Hosseini,et al.  A novel fuzzy facial expression recognition system based on facial feature extraction from color face images , 2012, Eng. Appl. Artif. Intell..

[11]  Mark Elshaw,et al.  Deep Learning for Illumination Invariant Facial Expression Recognition , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[12]  Zahir M. Hussain,et al.  Automatic facial expression recognition: feature extraction and selection , 2010, Signal, Image and Video Processing.

[13]  Lijun Yin,et al.  Facial Expression Recognition by De-expression Residue Learning , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[14]  Anil K. Jain,et al.  Face Detection in Color Images , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  B. Radig,et al.  Cross-database evaluation for facial expression recognition , 2014, Pattern Recognition and Image Analysis.

[16]  Matti Pietikäinen,et al.  Facial expression recognition from near-infrared videos , 2011, Image Vis. Comput..

[17]  Chung-Lin Huang,et al.  Facial Expression Recognition Using Model-Based Feature Extraction and Action Parameters Classification , 1997, J. Vis. Commun. Image Represent..

[18]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[19]  J. L. Crowley,et al.  Facial expression analysis and the affect space , 2015, Pattern Recognition and Image Analysis.

[20]  Uros Mlakar,et al.  Automated facial expression recognition based on histograms of oriented gradient feature vector differences , 2015, Signal Image Video Process..

[21]  Michael J. Lyons,et al.  Coding facial expressions with Gabor wavelets , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[22]  Ioannis Pitas,et al.  Comparison of ICA approaches for facial expression recognition , 2009, Signal Image Video Process..

[23]  D. J. Kim,et al.  Facial expression recognition using ASM-based post-processing technique , 2016, Pattern Recognition and Image Analysis.

[24]  D. Lundqvist,et al.  Karolinska Directed Emotional Faces , 2015 .

[25]  Mohamed A. Deriche,et al.  Scale-Space Properties of the Multiscale Morphological Dilation-Erosion , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Meng Wang,et al.  Individual-free representation-based classification for facial expression recognition , 2017, Signal Image Video Process..

[27]  Sazali Yaacob,et al.  Development of a personified face emotion recognition technique using fitness function , 2007, Artificial Life and Robotics.

[28]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[29]  Abdenour Bouzouane,et al.  User action and facial expression recognition for error detection system in an ambient assisted environment , 2018, Expert Syst. Appl..

[30]  Na Liu,et al.  Super Wide Regression Network for Unsupervised Cross-Database Facial Expression Recognition , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[31]  Siu-Yeung Cho,et al.  A face emotion tree structure representation with probabilistic recursive neural network modeling , 2010, Neural Computing and Applications.

[32]  Guangcan Liu,et al.  Deeper cascaded peak-piloted network for weak expression recognition , 2018, The Visual Computer.

[33]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).