4D facial expression recognition

In this paper, we focus on discrete expression classification using dynamic 3D sequences (4D data) recording the facial movements. A robust approach for registering 4D data is proposed and a variant of local binary patterns on three orthogonal planes is used for feature extraction. We present a fully automatic facial expression recognition pipeline. The system was evaluated on the publicly available facial expression database BU-4DFE and promising results were obtained.

[1]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Lijun Yin,et al.  Facial Expression Recognition Based on 3D Dynamic Range Model Sequences , 2008, ECCV.

[3]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[4]  J. Bourgain.  On lipschitz embedding of finite metric spaces in Hilbert space , 1985 .

[5]  Sotiris Malassiotis,et al.  Real-time 2D+3D facial action and expression recognition , 2010, Pattern Recognit..

[6]  Matti Pietikäinen,et al.  Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  R. Horaud,et al.  Surface feature detection and description with applications to mesh matching , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[9]  Andrew E. Johnson,et al.  Surface matching for object recognition in complex three-dimensional scenes , 1998, Image Vis. Comput..

[10]  Jun Wang,et al.  3D Facial Expression Recognition Based on Primitive Surface Feature Distribution , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[11]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[12]  Luiz Velho,et al.  Automatic 3D Facial Expression Analysis in Videos , 2005, AMFG.

[13]  Rama Chellappa,et al.  Discriminant Analysis for Recognition of Human Face Images (Invited Paper) , 1997, AVBPA.

[14]  Lijun Yin,et al.  A high-resolution 3D dynamic facial expression database , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[15]  Leonidas J. Guibas,et al.  Robust single-view geometry and motion reconstruction , 2009, ACM Trans. Graph..

[16]  Beat Fasel,et al.  Automati Fa ial Expression Analysis: A Survey , 1999 .

[17]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Peter Schröder,et al.  Discrete conformal mappings via circle patterns , 2005, TOGS.

[19]  Lijun Yin,et al.  Generating Realistic Facial Expressions with Wrinkles for Model-Based Coding , 2001, Comput. Vis. Image Underst..

[20]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[21]  Chitra Dorai,et al.  COSMOS - A Representation Scheme for 3D Free-Form Objects , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Stefanos Zafeiriou,et al.  A dynamic approach to the recognition of 3D facial expressions and their temporal models , 2011, Face and Gesture 2011.

[23]  Hao Li,et al.  Realtime performance-based facial animation , 2011, ACM Trans. Graph..

[24]  Ioannis A. Kakadiaris,et al.  3D facial expression recognition: A perspective on promises and challenges , 2011, Face and Gesture 2011.

[25]  Ashraf A. Kassim,et al.  A novel approach to classification of facial expressions from 3D-mesh datasets using modified PCA , 2009, Pattern Recognit. Lett..

[26]  C. Goodall.  Procrustes methods in the statistical analysis of shape , 1991 .

[27]  Jun Wang,et al.  A 3D facial expression database for facial behavior research , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[28]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2009, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[30]  Lijun Yin,et al.  Recognizing partial facial action units based on 3D dynamic range data for facial expression recognition , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[31]  Lijun Yin,et al.  Automatic Registration of Vertex Correspondences for 3D Facial Expression Analysis , 2008, 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems.

[32]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[33]  Thomas S. Huang,et al.  Expression recognition from 3D dynamic faces using robust spatio-temporal shape features , 2011, Face and Gesture 2011.

[34]  Ioannis A. Kakadiaris,et al.  Intraclass Retrieval of Nonrigid 3D Objects: Application to Face Recognition , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Ioannis A. Kakadiaris,et al.  Three-Dimensional Face Recognition in the Presence of Facial Expressions: An Annotated Deformable Model Approach , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[37]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[38]  Ying Zilu,et al.  Combining LBP and Adaboost for facial expression recognition , 2008, 2008 9th International Conference on Signal Processing.

[39]  Lijun Yin,et al.  Analyzing Facial Expressions Using Intensity-Variant 3D Data For Human Computer Interaction , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[40]  Leonidas J. Guibas,et al.  Robust single-view geometry and motion reconstruction , 2009, SIGGRAPH 2009.

[41]  A. Johnson.  A Representation for 3D Surface Matching , 1997 .