Facial Emotion Recognition in Videos using HOG and LBP

Emotions are found using verbal and non-verbal cues by analyzing voices and facial expressions. Monitoring emotional patterns of human is gaining importance in predicting the mood of a person. Facial emotion recognition is the process of detecting and recognizing different types of emotions in humans using facial expressions. The various steps include detection of the face and its landmarks, feature extraction of facial landmarks, and emotional state classification. The Haar cascading approach is used to detect different facial components such as eyes, mouth, and nose in an image. Facial features are analyzed using Histogram of Gradients (HOG) and Local Binary Pattern (LBP). The resultant feature vector is formed from the feature points. The three emotional states namely happy, sad and angry are classified using neural network classifier. The new feature points of test data are compared against trained data and their corresponding label values are displayed as the output for emotion recognition with the accuracy of 87% and 64% is being achieved using HOG and LBP techniques.

[1]  K. L. Shunmuganathan,et al.  Automatic emotion recognition in video , 2014, 2014 International Conference on Green Computing Communication and Electrical Engineering (ICGCCEE).

[2]  Sanjay V. Dudul,et al.  Neural Network Classifier for Human Emotion Recognition from Facial Expressions Using Discrete Cosine Transform , 2008, 2008 First International Conference on Emerging Trends in Engineering and Technology.

[3]  Ninad Thakoor,et al.  Facial emotion recognition in continuous video , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[4]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[5]  Hazem Abbas,et al.  Emotion Recognition by Facial Features using Recurrent Neural Networks , 2018, 2018 13th International Conference on Computer Engineering and Systems (ICCES).

[6]  Anand Singh Jalal,et al.  A multi-level classification approach for facial emotion recognition , 2012, 2012 IEEE International Conference on Computational Intelligence and Computing Research.

[7]  Anthony Choi,et al.  Facial Emotion Recognition Using Fuzzy Systems , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[8]  B. K. Julsing,et al.  Face Recognition with Local Binary Patterns , 2012 .

[9]  M. Turk,et al.  A simple, real-time range camera , 1989, Proceedings CVPR '89: IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Kwang-Eun Ko,et al.  Development of the facial feature extraction and emotion recognition method based on ASM and Bayesian network , 2009, 2009 IEEE International Conference on Fuzzy Systems.

[11]  Michael J. Lyons,et al.  Coding facial expressions with Gabor wavelets , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[12]  Burhan Ergen,et al.  Facial emotion recognition on a dataset using convolutional neural network , 2017, 2017 International Artificial Intelligence and Data Processing Symposium (IDAP).

[13]  Chandran Saravanan,et al.  Advancements and recent trends in emotion recognition using facial image analysis and machine learning models , 2017, 2017 International Conference on Electrical, Electronics, Communication, Computer, and Optimization Techniques (ICEECCOT).

[14]  Robert Sabourin,et al.  Adaptive appearance model tracking for still-to-video face recognition , 2016, Pattern Recognit..

[15]  Neelum Mehta,et al.  Facial Emotion recognition using Log Gabor filter and PCA , 2016, 2016 International Conference on Computing Communication Control and automation (ICCUBEA).

[16]  Goutam Sanyal,et al.  Facial emotion analysis using deep convolution neural network , 2017, 2017 International Conference on Signal Processing and Communication (ICSPC).

[17]  Kwang-Seok Hong,et al.  A study on emotion recognition method and its application using face image , 2017, 2017 International Conference on Information and Communication Technology Convergence (ICTC).

[18]  Kartika Candra Kirana,et al.  Facial Emotion Recognition Based on Viola-Jones Algorithm in the Learning Environment , 2018, 2018 International Seminar on Application for Technology of Information and Communication.

[19]  A. Majkowski,et al.  Analysis of Facial Features for the Use of Emotion Recognition , 2018, 19th International Conference Computational Problems of Electrical Engineering.

[20]  Peter Robinson,et al.  Expression training for complex emotions using facial expressions and head movements , 2015, 2015 International Conference on Affective Computing and Intelligent Interaction (ACII).

[21]  Mounir Sayadi,et al.  Feature points tracking and emotion classification , 2016, 2016 2nd International Conference on Advanced Technologies for Signal and Image Processing (ATSIP).

[22]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[23]  Milica M. Janković,et al.  Real-time Algorithms for Facial Emotion Recognition: A Comparison of Different Approaches , 2018, 2018 14th Symposium on Neural Networks and Applications (NEUREL).

[24]  Mansour Sheikhan,et al.  A fuzzy approach for facial emotion recognition , 2013, 2013 13th Iranian Conference on Fuzzy Systems (IFSC).

[25]  Bharati A. Dixit,et al.  Statistical moments based facial expression analysis , 2015, 2015 IEEE International Advance Computing Conference (IACC).

[26]  Abir Hudait,et al.  Automatic emotion detection model from facial expression , 2016, 2016 International Conference on Advanced Communication Control and Computing Technologies (ICACCCT).