Edge-Computing Convolutional Neural Network with Homography-Augmented Data for Facial Emotion Recognition

We design a convolutional neural network with homographyaugmented data to deal with facial emotion recognition applications. Different to other convolutional neural networks, our AsicNet is well-designed for embedded CPU and even aiming for ASIC, such as Intel Movidius VPU. We adjust the architecture of our AsicNet and train our AsicNet on the GPU server, meanwhile we consider the computation costs of embedded systems and ASICs. Moreover, we reconsider the deep learning flow and train the homography-augmented data so as to reach higher accuracy. Experimental results on both FER2013 anf JAFFE face datasets show that our AsicNet can not only have high accuracy (72.42% on FER2013; 99.82% on JAFFE) as compared to the state-of-arts but also reach 41.22 millisecond (24.26 FPS) on the embedded CPU and 15.25 millisecond (65.57 FPS) on Intel Movidius VPU to tell the facial emotion from a face image.

[1]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[2]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Michael J. Lyons,et al.  Coding facial expressions with Gabor wavelets , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[4]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[5]  Soo-Young Lee,et al.  Fusing Aligned and Non-aligned Face Information for Automatic Affect Recognition in the Wild: A Deep Learning Approach , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[6]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[7]  Hatice Gunes,et al.  CNN-based Facial Affect Analysis on Mobile Devices , 2018, ArXiv.

[8]  Yann LeCun,et al.  Convolutional networks and applications in vision , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[9]  Jun-Cheol Park,et al.  A Real-time Facial Expression Recognizer using Deep Neural Network , 2016, IMCOM.

[10]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[11]  Berin Martini,et al.  Hardware accelerated convolutional neural networks for synthetic vision systems , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[12]  Martin Kampel,et al.  Facial Expression Recognition using Convolutional Neural Networks: State of the Art , 2016, ArXiv.

[13]  Radhika M. Pai,et al.  Automatic Facial Expression Recognition Using DCNN , 2016 .

[14]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[15]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[17]  Yichuan Tang,et al.  Deep Learning using Support Vector Machines , 2013, ArXiv.

[18]  Davis E. King,et al.  Dlib-ml: A Machine Learning Toolkit , 2009, J. Mach. Learn. Res..

[19]  Ping Liu,et al.  Facial Expression Recognition via a Boosted Deep Belief Network , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Dewi Yanti Liliana,et al.  Enhancing CNN with Preprocessing Stage in Automatic Emotion Recognition , 2017, ICCSCI.

[21]  Uros Mlakar,et al.  Automated facial expression recognition based on histograms of oriented gradient feature vector differences , 2015, Signal Image Video Process..