Accurate computing of facial expression recognition using a hybrid feature extraction technique

Facial expression recognition (FER) serves as an essential tool for understanding human emotional behaviors. Facial expressions provide a wealth of information about intentions, emotions, and other inner states. Over the past two decades, the development of an automatic FER device has become one of the most demanding multimedia research areas in human–computer interaction systems. Several automatic systems have been introduced and have achieved precise identification accuracies. Due to the complex nature of the human face, however, problems still exist. Researchers are still struggling to develop effective methods for extracting features from images because of unclear features. This work proposes a methodology that improves high-performance computing in terms of the facial expression recognition accuracy. To achieve the goal of high accuracy, a hybrid method is proposed using the dual-tree m-band wavelet transform (DTMBWT) algorithm based on energy, entropy, and gray-level co-occurrence matrix (GLCM). It is accompanied by the use of a Gaussian mixture model (GMM) as the classification scheme to provide efficient identification of database images in terms of facial expressions. Using the DTMBWT, it is possible to derive many expression features from decomposition levels 1 to 6. Moreover, along with the GLCM features, the contrast and homogeneity features can be retrieved. All the features are eventually categorized and recognized with the aid of the GMM classifier. The proposed algorithms are tested using Japanese Female Facial Expression (JAFFE) database with seven different facial expressions: happiness, sadness, anger, fear, neutral, surprise, and disgust. The results of the experiments show that the highest precision of the proposed technique is 99.53%, which is observed at the 4th decomposition level of the DTMBWT.

[1]  Suneeta Agarwal,et al.  GLCM and its application in pattern recognition , 2017, 2017 5th International Symposium on Computational and Business Intelligence (ISCBI).

[2]  Byungyong Ryu,et al.  Local Directional Ternary Pattern for Facial Expression Recognition , 2017, IEEE Transactions on Image Processing.

[3]  Myung Jin Chung,et al.  Extension of cascaded simple feature based face detection to facial expression recognition , 2008, Pattern Recognit. Lett..

[4]  Janez Brest,et al.  Multi-Objective Differential Evolution for feature selection in Facial Expression Recognition systems , 2017, Expert Syst. Appl..

[5]  Peng Shan,et al.  A Micro-expression Recognition Algorithm for Students in Classroom Learning Based on Convolutional Neural Network , 2019, Traitement du Signal.

[6]  Kamalesh Kumar Sharma,et al.  Improved facial expression recognition using graph signal processing , 2017 .

[7]  Jian-Jiun Ding,et al.  Exemplar-embed complex matrix factorization for facial expression recognition , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[8]  Byungyong Ryu,et al.  Local Directional Ternary Pattern for Facial Expression Recognition. , 2017, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[9]  David G. Stork,et al.  Pattern classification, 2nd Edition , 2000 .

[10]  Sakshi Sharma,et al.  An improved method for facial expression recognition using hybrid approach of CLBP and Gabor filter , 2017, 2017 International Conference on Computing, Communication and Automation (ICCCA).

[11]  Jake K. Aggarwal,et al.  Facial expression recognition with temporal modeling of shapes , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[12]  Linda G. Shapiro,et al.  Computer and Robot Vision , 1991 .

[13]  Karim Faez,et al.  Facial expression recognition using dual dictionary learning , 2017, J. Vis. Commun. Image Represent..

[14]  Michael J. Lyons,et al.  Coding facial expressions with Gabor wavelets , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[15]  Liangnian Jin,et al.  Facial expression recognition via sparse representation using positive and reverse templates , 2016, IET Image Process..

[16]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[17]  D KeerthiAnandV WAVELETS FOR SPEAKER RECOGNITION USING GMM CLASSIFIER , 2017 .

[18]  Baoqing Li,et al.  Facial Expression Recognition From Image Sequence Based on LBP and Taylor Expansion , 2017, IEEE Access.

[19]  Haifeng Hu,et al.  Facial expression recognition with FRR-CNN , 2017 .

[20]  Sirion Vittayakorn,et al.  Facial expression recognition using local Gabor filters and PCA plus LDA , 2017, 2017 9th International Conference on Information Technology and Electrical Engineering (ICITEE).

[21]  Radu Tudor Ionescu,et al.  Local Learning With Deep and Handcrafted Features for Facial Expression Recognition , 2018, IEEE Access.

[22]  Yixiang Chen,et al.  Recognition of facial expression based on CNN-CBP features , 2017, 2017 29th Chinese Control And Decision Conference (CCDC).

[23]  Yong Du,et al.  Facial Expression Recognition Based on Deep Evolutional Spatial-Temporal Networks , 2017, IEEE Transactions on Image Processing.

[24]  Tong Zhang,et al.  A Deep Neural Network-Driven Feature Learning Method for Multi-view Facial Expression Recognition , 2016, IEEE Transactions on Multimedia.

[25]  Hongying Meng,et al.  Real time automated facial expression recognition app development on smart phones , 2017, 2017 8th IEEE Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON).

[26]  Kamlesh Mistry,et al.  Facial expression recongition using firefly-based feature optimization , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[27]  Li Cheng,et al.  Exploring a High-quality Outlying Feature Value Set for Noise-Resilient Outlier Detection in Categorical Data , 2018, CIKM.

[28]  Hao Wang,et al.  Facial expression recognition using iterative fusion of MO-HOG and deep features , 2018, The Journal of Supercomputing.

[29]  Manasi Patil,et al.  Facial Expression Recognition Based on Image Feature , 2012 .

[30]  Ivan W. Selesnick,et al.  The double-density dual-tree DWT , 2004, IEEE Transactions on Signal Processing.

[31]  Hamid R. Arabnia,et al.  Facial Expression Recognition Based on Fuzzy Networks , 2016, 2016 International Conference on Computational Science and Computational Intelligence (CSCI).

[32]  Rongfang Bie,et al.  Automatic facial expression recognition based on a deep convolutional-neural-network structure , 2017, 2017 IEEE 15th International Conference on Software Engineering Research, Management and Applications (SERA).

[33]  Min Wu,et al.  Nesterov accelerated gradient descent-based convolution neural network with dropout for facial expression recognition , 2017, 2017 11th Asian Control Conference (ASCC).

[34]  Yong Man Ro,et al.  Intra-Class Variation Reduction Using Training Expression Images for Sparse Representation Based Facial Expression Recognition , 2014, IEEE Transactions on Affective Computing.

[35]  Rongrong Ni,et al.  Facial Expression Recognition Using Weighted Mixture Deep Neural Network Based on Double-Channel Facial Images , 2018, IEEE Access.

[36]  Hanan Ali Alrikabi,et al.  Efficient face and facial expression recognition model , 2016, 2016 International Conference on Computing Communication Control and automation (ICCUBEA).

[37]  Jyoti Kumari,et al.  Facial Expression Recognition: A Survey , 2015 .

[38]  Asit Barman,et al.  Facial expression recognition using shape signature feature , 2017, 2017 Third International Conference on Research in Computational Intelligence and Communication Networks (ICRCICN).

[39]  Hong-Yuan Mark Liao,et al.  Deep dictionary learning for fine-grained image classification , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[40]  R. K. Gnanamurthy,et al.  A proposed method for the improvement in biometric facial image recognition using document-based classification , 2018, The Journal of Supercomputing.

[41]  Taner Cevik,et al.  A Survey of multimedia streaming in wireless sensor networks: progress, issues and design challenges , 2015, ArXiv.

[42]  Chun-Hsiung Tseng,et al.  A camera-based attention level assessment tool designed for classroom usage , 2017, The Journal of Supercomputing.

[43]  Tessy Mathew,et al.  Facial expression recognition and emotion classification system for sentiment analysis , 2017, 2017 International Conference on Networks & Advances in Computational Technologies (NetACT).

[44]  Haifeng Hu,et al.  Modified classification and regression tree for facial expression recognition with using difference expression images , 2017 .

[45]  Asit Barman,et al.  Texture signature based facial expression recognition using NARX , 2017, 2017 IEEE Calcutta Conference (CALCON).

[46]  Nicu Sebe,et al.  Learning Personalized Models for Facial Expression Analysis and Gesture Recognition , 2016, IEEE Transactions on Multimedia.

[47]  Ajay Khunteta,et al.  Facial expression recognition using Gabor filter and multi-layer artificial neural network , 2017, 2017 International Conference on Information, Communication, Instrumentation and Control (ICICIC).

[48]  Xiaodong Wang,et al.  Facial Expression Recognition: A Survey , 2019, Symmetry.

[49]  Caroline Chaux,et al.  Image analysis using a dual-tree M-band wavelet transform , 2006, IEEE Transactions on Image Processing.

[50]  Sandeep Sharma,et al.  A systematic survey of facial expression recognition techniques , 2017, 2017 International Conference on Computing Methodologies and Communication (ICCMC).

[51]  Qingguo Zhou,et al.  Intelligent monitor system based on cloud and convolutional neural networks , 2017, The Journal of Supercomputing.

[52]  Hong Zhang,et al.  Facial expression recognition via learning deep sparse autoencoders , 2018, Neurocomputing.

[53]  Qirong Mao,et al.  Hierarchical Bayesian Theme Models for Multipose Facial Expression Recognition , 2017, IEEE Transactions on Multimedia.