Automatic 3D facial expression recognition based on polytypic Local Binary Pattern

Most of the existing methods adopted in 3D facial expression recognition require manual-intervention, which should be avoided in automatic systems. In this paper, we propose an automatic data normalization method based on the point-to-point mapping between the depth value and the texture information of the facial data. Based on this, we further construct a new kind of 3D facial expression feature, namely polytypic Local Binary Patterns (p-LBP), which involves both the irregular divisions to keep the integrity of local faces and the fusion of depth and texture information of 3D models to enhance the recognition accuracy. The proposed strategy is tested on BU-3DFE database, and three kinds of classifiers are employed. Their remarkable results outperform the state of the art, and show the effectiveness of proposed features for 3D facial expression recognition. Therefore, p-LBP is validated and its simplicity opens a promising direction for automatic 3D facial expression recognition.

[1]  Emmanuel Dellandréa,et al.  Automatic 3D Facial Expression Recognition Based on a Bayesian Belief Net and a Statistical Facial Feature Model , 2010, 2010 20th International Conference on Pattern Recognition.

[2]  Jun Wang,et al.  A 3D facial expression database for facial behavior research , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[3]  H. Demirel,et al.  3D facial expression recognition with geometrically localized facial features , 2008, 2008 23rd International Symposium on Computer and Information Sciences.

[4]  Jian Sun,et al.  Face Alignment by Explicit Shape Regression , 2012, International Journal of Computer Vision.

[5]  Xiaoou Tang,et al.  Automatic facial expression recognition on a single 3D face by exploring shape deformation , 2009, ACM Multimedia.

[6]  M. Iqbal Saripan,et al.  3D facial expression recognition using maximum relevance minimum redundancy geometrical features , 2012, EURASIP Journal on Advances in Signal Processing.

[7]  Ashraf A. Kassim,et al.  Resampling Approach to Facial Expression Recognition Using 3D Meshes , 2010, 2010 20th International Conference on Pattern Recognition.

[8]  Di Huang,et al.  Local Binary Patterns and Its Application to Facial Image Analysis: A Survey , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[9]  Thomas S. Huang,et al.  3D facial expression recognition based on automatically selected features , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[10]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Thomas S. Huang,et al.  3D facial expression recognition based on properties of line segments connecting facial feature points , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[12]  Alberto Del Bimbo,et al.  A Set of Selected SIFT Features for 3D Facial Expression Recognition , 2010, 2010 20th International Conference on Pattern Recognition.

[13]  Yanfeng Sun,et al.  3D face recognition using local binary patterns , 2013, Signal Process..

[14]  Jun Wang,et al.  3D Facial Expression Recognition Based on Primitive Surface Feature Distribution , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[15]  Ashraf A. Kassim,et al.  A novel approach to classification of facial expressions from 3D-mesh datasets using modified PCA , 2009, Pattern Recognit. Lett..