Facial Expression Recognition Using 3D Convolutional Neural Network

This paper is concerned with video-based facial expression recognition frequently used in conjunction with HRI (Human-Robot Interaction) that can naturally interact between human and robot. For this purpose, we design a 3D-CNN(3D Convolutional Neural Networks) by augmenting dimensionality reduction methods such as PCA(Principal Component Analysis) and TMPCA(Tensor-based Multilinear Principal Component Analysis) to recognize simultaneously the successive frames with facial expression images obtained through video camera. The 3D-CNN can achieve some degree of shift and deformation invariance using local receptive fields and spatial subsampling through dimensionality reduction of redundant CNN’s output. The experimental results on video-based facial expression database reveal that the presented method shows a good performance in comparison to the conventional methods such as PCA and TMPCA.

[1]  Gang Lv,et al.  Recognition of Multi-Fontstyle Characters Based on Convolutional Neural Network , 2011, 2011 Fourth International Symposium on Computational Intelligence and Design.

[2]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[3]  Young Hoon Joo,et al.  Facial image analysis algorithm for emotion recognition , 2004 .

[4]  Ching Y. Suen,et al.  A novel hybrid CNN-SVM classifier for recognizing handwritten digits , 2012, Pattern Recognit..

[5]  Eun Yi Kim,et al.  Automatic Facial Expression Recognition using Tree Structures for Human Computer Interaction , 2007 .

[6]  Ah Chung Tsoi,et al.  Face recognition: a convolutional neural-network approach , 1997, IEEE Trans. Neural Networks.

[7]  Zhengyou Zhang,et al.  Comparison between geometry-based and Gabor-wavelets-based facial expression recognition using multi-layer perceptron , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[8]  A. Ganapathiraju,et al.  LINEAR DISCRIMINANT ANALYSIS - A BRIEF TUTORIAL , 1995 .

[9]  Takeo Kanade,et al.  Detection, tracking, and classification of action units in facial expression , 2000, Robotics Auton. Syst..

[10]  Kenji Mase,et al.  Recognition of Facial Expression from Optical Flow , 1991 .

[11]  D. Mitchell Wilkes,et al.  An application of passive human-robot interaction: human tracking based on attention distraction , 2002, IEEE Trans. Syst. Man Cybern. Part A.

[12]  Y. LeCun,et al.  Learning methods for generic object recognition with invariance to pose and lighting , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[13]  P. Ekman.  Unmasking The Face , 1975 .

[14]  Yasunari Yoshitomi,et al.  A method for detecting transitions of emotional states using a thermal facial image based on a synthesis of facial expressions , 2000, Robotics Auton. Syst..

[15]  Ming Xie,et al.  Finger identification and hand posture recognition for human-robot interaction , 2007, Image Vis. Comput..

[16]  Ming Yang,et al.  3D Convolutional Neural Networks for Human Action Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[18]  Masakazu Matsugu,et al.  Subject independent facial expression recognition with robust face detection using a convolutional neural network , 2003, Neural Networks.

[19]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[20]  Hubert Cecotti,et al.  Convolutional Neural Network with embedded Fourier Transform for EEG classification , 2008, 2008 19th International Conference on Pattern Recognition.

[21]  Myung-Geun Chun,et al.  Facial Expression Recognition using ICA-Factorial Representation Method , 2003 .

[22]  Gérard G. Medioni,et al.  Robust real-time vision for a personal service robot , 2007, Comput. Vis. Image Underst..