Real Time Emotion Recognition from Facial Expressions Using CNN Architecture

Emotion is an important topic in different fields such as biomedical engineering, psychology, neuroscience and health. Emotion recognition could be useful for diagnosis of brain and psychological disorders. In recent years, deep learning has progressed much in the field of image classification. In this study, we proposed a Convolutional Neural Network (CNN) based LeNet architecture for facial expression recognition. First of all, we merged 3 datasets (JAFFE, KDEF and our custom dataset). Then we trained our LeNet architecture for emotion states classification. In this study, we achieved accuracy of 96.43% and validation accuracy of 91.81% for classification of 7 different emotions through facial expressions.

[1]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[2]  Quan Wen,et al.  Deep Learning Models for Facial Expression Recognition , 2018, 2018 Digital Image Computing: Techniques and Applications (DICTA).

[3]  Nikolaos Christou,et al.  Human Facial Expression Recognition with Convolution Neural Networks , 2018, Advances in Intelligent Systems and Computing.

[4]  Robert C. Roberts,et al.  What an Emotion is: A Sketch , 1988 .

[5]  D. Lundqvist,et al.  Karolinska Directed Emotional Faces , 2015 .

[6]  K. LaBar,et al.  Regulating emotion through distancing: A taxonomy, neurocognitive model, and supporting meta-analysis , 2019, Neuroscience & Biobehavioral Reviews.

[7]  M. Cabanac.  What is emotion? , 2002, Behavioural Processes.

[8]  Aydin Akan,et al.  Emotion Recognition from EEG Signals by Using Empirical Mode Decomposition , 2018, 2018 Medical Technologies National Congress (TIPTEKNO).

[9]  Jianfeng Zhao,et al.  Speech emotion recognition using deep 1D & 2D CNN LSTM networks , 2019, Biomed. Signal Process. Control..

[10]  Bryan T. Denny,et al.  Negative affect mediates the relationship between use of emotion regulation strategies and general health in college-aged students , 2019 .

[11]  Guihua Wen,et al.  Facial Expression Recognition Based on Complexity Perception Classification Algorithm , 2018, ArXiv.

[12]  Chung-Hsien Wu,et al.  Speech Emotion Recognition Using Deep Neural Network Considering Verbal and Nonverbal Speech Sounds , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[13]  Ruyi Xu,et al.  Automatic social signal analysis: Facial expression recognition using difference convolution neural network , 2019, J. Parallel Distributed Comput..

[14]  Ming Li,et al.  Facial Expression Recognition Algorithm Based on the Combination of CNN and K-Means , 2019, ICMLC '19.

[15]  Andrew Zisserman,et al.  Emotion Recognition in Speech using Cross-Modal Transfer in the Wild , 2018, ACM Multimedia.

[16]  Aminda J. O'Hare,et al.  Emotion words: Facing change , 2018, Journal of Experimental Social Psychology.

[17]  Chris Bowerman,et al.  Human Centric Facial Expression Recognition , 2018 .

[18]  Michael J. Lyons,et al.  Coding facial expressions with Gabor wavelets , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[19]  Mohammad Shahadat Hossain,et al.  Facial Expression Recognition using Convolutional Neural Network with Data Augmentation , 2019, 2019 Joint 8th International Conference on Informatics, Electronics & Vision (ICIEV) and 2019 3rd International Conference on Imaging, Vision & Pattern Recognition (icIVPR).

[20]  Soo-Hyung Kim,et al.  Facial Emotion Recognition Using an Ensemble of Multi-Level Convolutional Neural Networks , 2019, Int. J. Pattern Recognit. Artif. Intell..