A Smile Detection Method Based on Improved LeNet-5 and Support Vector Machine

Conventional facial expression recognition methods usually deal with frontal face images via only one or several features, which are easy to loss useful information and sensitive to face poses, scales and noise. As an interesting application of facial expression, this paper proposes an effective smile detection method under unconstrained scenarios, employing convolution neural network to learn and automatically extract discriminative features from a large number of human face images. Specifically, our method firstly converts the original color images to grayscale images, and due to the important role of mouth in expression analysis, we then localizes the mouth region according to 5 key points on the face. After the brightness adjustment and size normalization, the mouth images are input as training images of an improved LeNet-5 model to learn and automatically extract the discriminative features of the mouth regions. Finally, a SVM classifier is trained to distinguish smiling or non-smiling. Experimental results of the public MTFL database and GENKI-4K database show that the accuracy rates of our method are up to 87.81% and 86.80%, respectively.

[1]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[3]  Ping Liu,et al.  Facial Expression Recognition via a Boosted Deep Belief Network , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Caifeng Shan,et al.  Smile detection by boosting pixel differences , 2012, IEEE Transactions on Image Processing.

[5]  Marcus Liwicki,et al.  DeXpression: Deep Convolutional Neural Network for Expression Recognition , 2015, ArXiv.

[6]  Hong Liu,et al.  A new descriptor of gradients Self-Similarity for smile detection in unconstrained scenarios , 2016, Neurocomputing.

[7]  Xiaoou Tang,et al.  Facial Landmark Detection by Deep Multi-task Learning , 2014, ECCV.

[8]  Gwen Littlewort,et al.  Toward Practical Smile Detection , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Shiguang Shan,et al.  AU-aware Deep Networks for facial expression recognition , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[10]  Horst Bischof,et al.  Annotated Facial Landmarks in the Wild: A large-scale, real-world database for facial landmark localization , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[11]  Klaus-Robert Müller,et al.  Efficient BackProp , 2012, Neural Networks: Tricks of the Trade.

[12]  Dong Yu,et al.  Deep Learning: Methods and Applications , 2014, Found. Trends Signal Process..

[13]  Ivanna K. Timotius,et al.  Smile recognition system based on lip corners identification , 2014, 2014 The 1st International Conference on Information Technology, Computer, and Electrical Engineering.

[14]  Patrick O. Glauner.  Deep Convolutional Neural Networks for Smile Recognition , 2015, ArXiv.

[15]  Laurence T. Yang,et al.  Launching an Efficient Participatory Sensing Campaign , 2015, MM 2015.

[16]  Bir Bhanu,et al.  Efficient smile detection by Extreme Learning Machine , 2015, Neurocomputing.

[17]  Yu-Hao Huang.  FACE DETECTION AND SMILE DETECTION , 2009 .

[18]  Geoffrey Zweig,et al.  Recent advances in deep learning for speech research at Microsoft , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[19]  Xiaogang Wang,et al.  Deep Convolutional Network Cascade for Facial Point Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Lianwen Jin,et al.  A novel feature extraction method using Pyramid Histogram of Orientation Gradients for smile recognition , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).