Face and Facial Expressions Recognition and Analysis

Face recognition and facial expression analysis are essential abilities of humans, which provide the basic visual clues during human-computer interaction. It is important to enable the virtual human/social robot such capabilities in order to achieve autonomous behavior. Local binary pattern (LBP) has been widely used in face recognition and facial expression analysis. It is popular because of robustness to illumination variation and alignment error. However, local binary pattern still has some limitations, e.g. it is sensitive to image noise. Local ternary pattern (LTP), fuzzy LBP and many other LBP variants partially solve this problem. However, these approaches treat the corrupted image patterns as they are, and do not have an mechanism to recover the underlying patterns. In view of this, we develop a noise-resistant LBP to preserve the image micro-structures in presence of noise. We encode the small pixel difference as an uncertain state first, and then determine its value based on the other bits of the LBP code. Most image micro-structures are represented by uniform codes and non-uniform codes mainly represent noise patterns. Therefore, we assign the value of uncertain bit so as to form possible uniform codes. In such a way, we develop an error-correction mechanism to recover the distorted image patterns. In addition, we find that some image patterns such as lines are not captured in uniform codes. They represent a set of important local primitives for pattern recognition. We thus define an extended noise-resistant LBP (ENRLBP) to capture line patterns. NRLBP and ENRLBP are validated extensively on face recognition, facial expression analysis and other recognition tasks. They are shown more resistant to image noise compared with LBP, LTP and many other variants. These two approaches greatly enhance the performance of face recognition and facial expression analysis.

[1]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[2]  Xudong Jiang,et al.  A complete and fully automated face verification system on mobile devices , 2013, Pattern Recognit..

[3]  Raj Gupta,et al.  Robust order-based methods for feature description , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  Shuicheng Yan,et al.  Discriminative local binary patterns for human detection in personal album , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Matti Pietikäinen,et al.  Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Robert F. Murphy,et al.  A neural network classifier capable of recognizing the patterns of all major subcellular structures in fluorescence microscope images of HeLa cells , 2001, Bioinform..

[7]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Zhenhua Guo,et al.  A Completed Modeling of Local Binary Pattern Operator for Texture Classification , 2010, IEEE Transactions on Image Processing.

[9]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Zhongfei Zhang,et al.  Heat Kernel Based Local Binary Pattern for Face Representation , 2010, IEEE Signal Processing Letters.

[11]  Baochang Zhang,et al.  Sobel-LBP , 2008, 2008 15th IEEE International Conference on Image Processing.

[12]  Shu Liao,et al.  Dominant Local Binary Patterns for Texture Classification , 2009, IEEE Transactions on Image Processing.

[13]  Matti Pietikäinen,et al.  Outex - new framework for empirical evaluation of texture analysis algorithms , 2002, Object recognition supported by user interaction for service robots.

[14]  Shu-Yuan Chen,et al.  Retrieval of translated, rotated and scaled color textures , 2003, Pattern Recognit..

[15]  Shuicheng Yan,et al.  An HOG-LBP human detector with partial occlusion handling , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[16]  Wen Gao,et al.  Local Gabor binary pattern histogram sequence (LGBPHS): a novel non-statistical model for face representation and recognition , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[17]  A. Martínez,et al.  The AR face databasae , 1998 .

[18]  Guizhong Liu,et al.  Scale- and Rotation-Invariant Local Binary Pattern Using Scale-Adaptive Texton and Subuniform-Based Circular Shift , 2012, IEEE Transactions on Image Processing.

[19]  Matti Pietikäinen,et al.  Rotation-Invariant Image and Video Description With Local Binary Pattern Features , 2012, IEEE Transactions on Image Processing.

[20]  Bertrand Zavidovique,et al.  Median Binary Pattern for Textures Classification , 2007, ICIAR.

[21]  Matti Pietikäinen,et al.  Modeling pixel process with scale invariant local patterns for background subtraction in complex scenes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Loris Nanni,et al.  Local binary patterns variants as texture descriptors for medical image analysis , 2010, Artif. Intell. Medicine.

[23]  Xudong Jiang,et al.  Noise-Resistant Local Binary Pattern With an Embedded Error-Correction Mechanism , 2013, IEEE Transactions on Image Processing.

[24]  Loris Nanni,et al.  A simple method for improving local binary patterns by considering non-uniform patterns , 2012, Pattern Recognit..

[25]  Xiaoyang Tan,et al.  Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions , 2007, IEEE Transactions on Image Processing.

[26]  David R. Bull,et al.  Robust texture features for blurred images using Undecimated Dual-Tree Complex Wavelets , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[27]  Xudong Jiang,et al.  Relaxed local ternary pattern for face recognition , 2013, 2013 IEEE International Conference on Image Processing.

[28]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Ahmad Reza Naghsh-Nilchi,et al.  Noise tolerant local binary pattern operator for efficient texture analysis , 2012, Pattern Recognit. Lett..

[30]  Wei Wei,et al.  Centralized Binary Patterns Embedded with Image Euclidean Distance for Facial Expression Recognition , 2008, 2008 Fourth International Conference on Natural Computation.

[31]  Moulay A. Akhloufi,et al.  Locally adaptive texture features for multispectral face recognition , 2010, 2010 IEEE International Conference on Systems, Man and Cybernetics.

[32]  Xudong Jiang,et al.  Dynamic texture recognition using enhanced LBP features , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[33]  Wen-Hung Liao,et al.  Texture Classification Using Uniform Extended Local Ternary Patterns , 2010, 2010 IEEE International Symposium on Multimedia.

[34]  Stan Z. Li,et al.  Shape localization based on statistical method using extended local binary pattern , 2004, Third International Conference on Image and Graphics (ICIG'04).

[35]  Nick Cercone,et al.  Local Triplet Pattern for Content-Based Image Retrieval , 2009, ICIAR.

[36]  Xudong Jiang,et al.  Learning binarized pixel-difference pattern for scene recognition , 2013, 2013 IEEE International Conference on Image Processing.

[37]  Marko Heikkilä,et al.  Description of interest regions with local binary patterns , 2009, Pattern Recognit..

[38]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Jianxin Wu,et al.  mCENTRIST: A Multi-Channel Feature Generation Mechanism for Scene Categorization , 2014, IEEE Transactions on Image Processing.

[40]  James M. Rehg,et al.  CENTRIST: A Visual Descriptor for Scene Categorization , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Dimitris K. Iakovidis,et al.  Fuzzy binary patterns for uncertainty-aware texture representation , 2012 .

[42]  Max Q.-H. Meng,et al.  Texture analysis for ulcer detection in capsule endoscopy images , 2009, Image Vis. Comput..

[43]  Pod Hyb.  Extended Set of Local Binary Patterns for Rapid Object Detection , 2010 .

[44]  Cheng Wang,et al.  A novel extended local-binary-pattern operator for texture analysis , 2008, Inf. Sci..

[45]  Andreas Ernst,et al.  Face detection with the modified census transform , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[46]  Xudong Jiang,et al.  LBP-Based Edge-Texture Features for Object Recognition , 2014, IEEE Transactions on Image Processing.

[47]  Matti Pietikäinen,et al.  A Bayesian Local Binary Pattern texture descriptor , 2008, 2008 19th International Conference on Pattern Recognition.

[48]  Dimitrios K. Iakovidis,et al.  Fuzzy Local Binary Patterns for Ultrasound Texture Characterization , 2008, ICIAR.

[49]  Lei Huang,et al.  A new Probabilistic Local Binary Pattern for face verification , 2009, ICIP 2009.