Facial Expression Classification Using Convolutional Neural Network and Real Time Application

Facial expression is an important feature that gives information about a person's psychological situation. People use these expressions while communicating and socializing and they have lots of information related to inner world of an individual. Therefore it is important to understand the meaning of these facial expressions automatically and use this information. This paper presents a method for facial expression classification with grayscale images from Kaggle Face Dataset with a Convolutional Neural Network and a realtime user interface in order to test the performance online. Data augmentation is used to increase the diversity of the samples. Neural network is created with Matlab and user interface is created via App Designer. Different training and fine-tuning techniques are employed in the design. The overall accuracy 61.8% is achieved across seven different facial expression categories with test dataset supplied in Kaggle domain.

[1]  Edilson de Aguiar,et al.  A Facial Expression Recognition System Using Convolutional Networks , 2015, 2015 28th SIBGRAPI Conference on Graphics, Patterns and Images.

[2]  Yoshua Bengio,et al.  Object Recognition with Gradient-Based Learning , 1999, Shape, Contour and Grouping in Computer Vision.

[3]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[4]  A. Raghuvanshi,et al.  Facial Expression Recognition with Convolutional Neural Networks , 2016 .

[5]  George D. C. Cavalcanti,et al.  Facial expression Recognition based on Motion Estimation , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[6]  Edilson de Aguiar,et al.  Facial expression recognition with Convolutional Neural Networks: Coping with few data and the training sample order , 2017, Pattern Recognit..

[7]  K. Prodanova,et al.  Modeling data for tilted implants in grafted with bio-oss maxillary sinuses using logistic regression , 2014 .

[8]  Yoshua Bengio,et al.  Challenges in representation learning: A report on three machine learning contests , 2013, Neural Networks.

[9]  Anima Majumder,et al.  Facial expressions recognition system using Bayesian inference , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[10]  G. Cottrell,et al.  EMPATH: A Neural Network that Categorizes Facial Expressions , 2002, Journal of Cognitive Neuroscience.

[11]  Friedhelm Schwenker,et al.  A Hidden Markov Model Based Approach for Facial Expression Recognition in Image Sequences , 2010, ANNPR.

[12]  Craig A. Smith,et al.  From appraisal to emotion: Differences among unpleasant feelings , 1988 .

[13]  Kunihiko Fukushima,et al.  Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position , 1980, Biological Cybernetics.