Robust Facial Expression Recognition Based on Local Directional Pattern

Automatic facial expression recognition has many potential applications in different areas of human computer interaction. However, they are not yet fully realized due to the lack of an effective facial feature descriptor. In this paper, we present a new appearancebased feature descriptor, the local directional pattern (LDP), to represent facial geometry and analyze its performance in expression recognition. An LDP feature is obtained by computing the edge response values in 8 directions at each pixel and encoding them into an 8 bit binary number using the relative strength of these edge responses. The LDP descriptor, a distribution of LDP codes within an image or image patch, is used to describe each expression image. The effectiveness of dimensionality reduction techniques, such as principal component analysis and AdaBoost, is also analyzed in terms of computational cost saving and classification accuracy. Two well-known machine learning methods, template matching and support vector machine, are used for classification using the Cohn-Kanade and Japanese female facial expression databases. Better classification accuracy shows the superiority of LDP descriptor against other appearance-based feature descriptors.

[1]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Wen Gao,et al.  2D Cascaded AdaBoost for Eye Localization , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[3]  Oksam Chae,et al.  Local Directional Pattern (LDP) – A Robust Image Descriptor for Object Recognition , 2010, 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[4]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[5]  M. Pietikäinen,et al.  Facial Expression Recognition with Local Binary Patterns and Linear Programming 1 , 2005 .

[6]  Shaogang Gong,et al.  Robust facial expression recognition using local binary patterns , 2005, IEEE International Conference on Image Processing 2005.

[7]  Gwen Littlewort,et al.  Recognizing facial expression: machine learning and application to spontaneous behavior , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[8]  J. N. Bassili Emotion recognition: the role of facial movement and the relative importance of upper and lower areas of the face. , 1979, Journal of personality and social psychology.

[9]  Zhengyou Zhang,et al.  Comparison between geometry-based and Gabor-wavelets-based facial expression recognition using multi-layer perceptron , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[10]  Oksam Chae,et al.  Robust Facial Expression Recognition Based on Local Directional Pattern , 2010 .

[11]  Takahiro Okabe,et al.  Gaze Estimation from Low Resolution Images , 2006, PSIVT.

[12]  Frank Y. Shih,et al.  Recognizing facial action units using independent component analysis and support vector machine , 2006, Pattern Recognit..

[13]  Sung-Jea Ko,et al.  Person identification system for future digital tv with intelligence , 2007, IEEE Transactions on Consumer Electronics.

[14]  Ying-li Tian,et al.  Evaluation of Face Resolution for Expression Analysis , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[15]  Michael J. Lyons,et al.  Automatic Classification of Single Facial Images , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Marian Stewart Bartlett,et al.  Face recognition by independent component analysis , 2002, IEEE Trans. Neural Networks.

[17]  Kyoung-Ho Choi,et al.  A Probabilistic Network for Facial Feature Verification , 2003 .

[18]  Yong Man Ro,et al.  Facial Feature Extraction Based on Private Energy Map in DCT Domain , 2007 .

[19]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[20]  Maja Pantic,et al.  Fully Automatic Facial Action Unit Detection and Temporal Analysis , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[21]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Oksam Chae,et al.  Local Directional Pattern (LDP) for face recognition , 2010, 2010 Digest of Technical Papers International Conference on Consumer Electronics (ICCE).

[23]  Yoram Singer,et al.  Improved Boosting Algorithms Using Confidence-rated Predictions , 1998, COLT' 98.

[24]  Matti Pietikäinen,et al.  Boosted multi-resolution spatiotemporal descriptors for facial expression recognition , 2009, Pattern Recognit. Lett..

[25]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Md. Zia Uddin,et al.  An enhanced independent component-based human facial expression recognition from video , 2009, IEEE Transactions on Consumer Electronics.

[27]  Vijayan K. Asari,et al.  Facial Recognition Using Multisensor Images Based on Localized Kernel Eigen Spaces , 2009, IEEE Transactions on Image Processing.

[28]  Matti Pietikäinen,et al.  Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Baochang Zhang,et al.  Sobel-LBP , 2008, 2008 15th IEEE International Conference on Image Processing.

[30]  Cheng Wang,et al.  A novel extended local-binary-pattern operator for texture analysis , 2008, Inf. Sci..

[31]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[32]  Maja Pantic,et al.  Facial Action Unit Detection using Probabilistic Actively Learned Support Vector Machines on Tracked Facial Point Data , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[33]  Hyeonjoon Moon,et al.  The FERET Evaluation Methodology for Face-Recognition Algorithms , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Takeo Kanade,et al.  Facial Expression Analysis , 2011, AMFG.

[35]  Cherukuri Aswani Kumar,et al.  Analysis of unsupervised dimensionality reduction techniques , 2009, Comput. Sci. Inf. Syst..

[36]  Beat Fasel,et al.  Automati Fa ial Expression Analysis: A Survey , 1999 .

[37]  Marian Stewart Bartlett,et al.  Classifying Facial Actions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Lisa M. Brown,et al.  Real World Real-time Automatic Recognition of Facial Expressions , 2003 .

[39]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[40]  Claudia Iancu,et al.  Biometric Access Control for Digital Media Streams in Home Networks , 2007, 2007 Digest of Technical Papers International Conference on Consumer Electronics.

[41]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[42]  Alex Pentland,et al.  Face recognition using eigenfaces , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.