Robust Facial Expression Recognition Based on Local Directional Pattern

Automatic facial expression recognition has many potential applications in different areas of human computer interaction. However, they are not yet fully realized due to the lack of an effective facial feature descriptor. In this paper, we present a new appearance-based feature descriptor, the local directional pattern (LDP), to represent facial geometry and analyze its performance in expression recognition. An LDP feature is obtained by computing the edge response values in 8 directions at each pixel and encoding them into an 8 bit binary number using the relative strength of these edge responses. The LDP descriptor, a distribution of LDP codes within an image or image patch, is used to describe each expression image. The effectiveness of dimensionality reduction techniques, such as principal component analysis and AdaBoost, is also analyzed in terms of computational cost saving and classification accuracy. Two well-known machine learning methods, template matching and support vector machine, are used for classification using the Cohn-Kanade and Japanese female facial expression databases. Better classification accuracy shows the superiority of LDP descriptor against other appearance-based feature descriptors.

[1]  Marian Stewart Bartlett,et al.  Face recognition by independent component analysis , 2002, IEEE Trans. Neural Networks.

[2]  Kyoung-Ho Choi,et al.  A Probabilistic Network for Facial Feature Verification , 2003 .

[3]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[4]  Takahiro Okabe,et al.  Gaze Estimation from Low Resolution Images , 2006, PSIVT.

[5]  M. Pietikäinen,et al.  FACIAL EXPRESSION RECOGNITION WITH LOCAL BINARY PATTERNS AND LINEAR PROGRAMMING , 2004 .

[6]  Lisa M. Brown,et al.  Real World Real-time Automatic Recognition of Facial Expressions , 2003 .

[7]  Gwen Littlewort,et al.  Recognizing facial expression: machine learning and application to spontaneous behavior , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[8]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Yong Man Ro,et al.  Facial Feature Extraction Based on Private Energy Map in DCT Domain , 2007 .

[11]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[12]  J. N. Bassili Emotion recognition: the role of facial movement and the relative importance of upper and lower areas of the face. , 1979, Journal of personality and social psychology.

[13]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Md. Zia Uddin,et al.  An enhanced independent component-based human facial expression recognition from video , 2009, IEEE Transactions on Consumer Electronics.

[15]  Shaogang Gong,et al.  Robust facial expression recognition using local binary patterns , 2005, IEEE International Conference on Image Processing 2005.

[16]  Frank Y. Shih,et al.  Recognizing facial action units using independent component analysis and support vector machine , 2006, Pattern Recognit..

[17]  Sung-Jea Ko,et al.  Person identification system for future digital tv with intelligence , 2007, IEEE Transactions on Consumer Electronics.

[18]  Ying-li Tian,et al.  Evaluation of Face Resolution for Expression Analysis , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[19]  Matti Pietikäinen,et al.  Boosted multi-resolution spatiotemporal descriptors for facial expression recognition , 2009, Pattern Recognit. Lett..

[20]  Maja Pantic,et al.  Facial Action Unit Detection using Probabilistic Actively Learned Support Vector Machines on Tracked Facial Point Data , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[21]  Vijayan K. Asari,et al.  Facial Recognition Using Multisensor Images Based on Localized Kernel Eigen Spaces , 2009, IEEE Transactions on Image Processing.

[22]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[23]  Cherukuri Aswani Kumar,et al.  Analysis of unsupervised dimensionality reduction techniques , 2009, Comput. Sci. Inf. Syst..

[24]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[25]  Fernando De la Torre,et al.  Facial Expression Analysis , 2011, Visual Analysis of Humans.

[26]  Oksam Chae,et al.  Local Directional Pattern (LDP) for face recognition , 2010, 2010 Digest of Technical Papers International Conference on Consumer Electronics (ICCE).

[27]  Beat Fasel,et al.  Automati Fa ial Expression Analysis: A Survey , 1999 .

[28]  Wen Gao,et al.  2D Cascaded AdaBoost for Eye Localization , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[29]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[30]  Seong-Whan Lee Off-Line Recognition of Totally Unconstrained Handwritten Numerals Using Multilayer Cluster Neural Network , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Ling Shao,et al.  Human Action Recognition Using LBP-TOP as Sparse Spatio-Temporal Feature Descriptor , 2009, CAIP.

[32]  Maja Pantic,et al.  Fully Automatic Facial Action Unit Detection and Temporal Analysis , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[33]  Yoram Singer,et al.  Improved Boosting Algorithms Using Confidence-rated Predictions , 1998, COLT' 98.

[34]  Claudia Iancu,et al.  Biometric Access Control for Digital Media Streams in Home Networks , 2007, 2007 Digest of Technical Papers International Conference on Consumer Electronics.

[35]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[36]  Cheng Wang,et al.  A novel extended local-binary-pattern operator for texture analysis , 2008, Inf. Sci..

[37]  Marian Stewart Bartlett,et al.  Classifying Facial Actions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  V. Kshirsagar,et al.  Face recognition using Eigenfaces , 2011, 2011 3rd International Conference on Computer Research and Development.

[39]  Michael J. Lyons,et al.  Automatic Classification of Single Facial Images , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  Matti Pietikäinen,et al.  Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[42]  Oksam Chae,et al.  Local Directional Pattern (LDP) – A Robust Image Descriptor for Object Recognition , 2010, 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[43]  Garrison W. Cottrell,et al.  Representing Face Images for Emotion Classification , 1996, NIPS.

[44]  Zhengyou Zhang,et al.  Comparison between geometry-based and Gabor-wavelets-based facial expression recognition using multi-layer perceptron , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.