Robust face recognition with structural binary gradient patterns

A compact and efficient face image descriptor, termed Binary Gradient Patterns (BGP), is proposed.Structural BGP extracts structural gradient patterns as binary strings in local regions for effective representation.BGP and its enhanced BGPM are shown robust to various variations, esp. illumination.Experimental results demonstrate significant performance improvements over the existing methods, including with deep networks. This paper presents a compact and efficient yet powerful binary framework based on image gradients for robust facial representation. It is termed as Binary Gradient Patterns (BGP). To discover underlying local structures in the gradient domain, image gradients are computed from multiple directions and encoded into a set of binary strings. Certain types of these binary strings have meaningful local structures and textures, as they detect micro oriented edges and retain strong local orientation, thus enabling great discrimination. Face representations by these structural BGP histograms exhibit profound robustness against various facial image variations, in particular illumination. The binary strategy realized by local correlations substantially simplifies the computational complexity and achieves extremely efficient processing with only 0.0032s in Matlab for a typical image. Furthermore, the discrimination power of the BGP has been enhanced on a set of orientations of the image-gradient magnitudes. Extensive experimental results on various benchmarks demonstrate that the BGP-based representations significantly improve over the existing local descriptors and state-of-the-art methods in the terms of discrimination, robustness and complexity and in many cases the improvements are substantial. Combining with the deep networks, the proposed descriptors can further improve the performance of the deep networks on real-world datasets. Matlab codes for the BGP-based descriptors are available at: http://www.eee.manchester.ac.uk/our-research/research-groups/sisp/research-areas/vip/software/.

[1]  Jian-Huang Lai,et al.  Normalization of Face Illumination Based on Large-and Small-Scale Features , 2011, IEEE Transactions on Image Processing.

[2]  Matti Pietikäinen,et al.  Face Recognition by Exploring Information Jointly in Space, Scale and Orientation , 2011, IEEE Transactions on Image Processing.

[3]  Mohammed Bennamoun,et al.  Linear Regression for Face Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Xiaoou Tang,et al.  Surpassing Human-Level Face Verification Performance on LFW with GaussianFace , 2014, AAAI.

[5]  Matti Pietikäinen,et al.  IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2009, TPAMI-2008-09-0620 1 WLD: A Robust Local Image Descriptor , 2022 .

[6]  Jun Luo,et al.  Person-Specific SIFT Features for Face Recognition , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[7]  Wen Gao,et al.  Local Gabor binary pattern histogram sequence (LGBPHS): a novel non-statistical model for face representation and recognition , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[8]  Ioannis Rigas,et al.  Face recognition via local sparse coding , 2011, 2011 International Conference on Computer Vision.

[9]  Tal Hassner,et al.  Face recognition in unconstrained videos with matched background similarity , 2011, CVPR 2011.

[10]  Wen Gao,et al.  Histogram of Gabor Phase Patterns (HGPP): A Novel Object Representation Approach for Face Recognition , 2007, IEEE Transactions on Image Processing.

[11]  Jun Guo,et al.  In Defense of Sparsity Based Face Recognition , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  A. Martínez,et al.  The AR face databasae , 1998 .

[14]  Takeo Kanade,et al.  Multi-PIE , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[15]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  HeRan,et al.  Maximum Correntropy Criterion for Robust Face Recognition , 2011 .

[17]  Jun Guo,et al.  Gabor-Eigen-Whiten-Cosine: A Robust Scheme for Face Recognition , 2005, AMFG.

[18]  Dorin Comaniciu,et al.  Total variation models for variable lighting face recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Lei Zhang,et al.  Sparse representation or collaborative representation: Which helps face recognition? , 2011, 2011 International Conference on Computer Vision.

[20]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[21]  Ognjen Arandjelovic,et al.  Making the most of the self-quotient image in face recognition , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[22]  Roberto Cipolla,et al.  Achieving robust face recognition from video by combining a weak photometric model and a learnt generic face invariant , 2013, Pattern Recognit..

[23]  Yihong Gong,et al.  Locality-constrained Linear Coding for image classification , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Weilin Huang,et al.  A dissimilarity kernel with local features for robust facial recognition , 2010, 2010 IEEE International Conference on Image Processing.

[25]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[26]  Andrew Zisserman,et al.  Deep Face Recognition , 2015, BMVC.

[27]  Chu-Song Chen,et al.  Face Recognition and Retrieval Using Cross-Age Reference Coding With Cross-Age Celebrity Dataset , 2015, IEEE Transactions on Multimedia.

[28]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[29]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Tomaso A. Poggio,et al.  Face recognition: component-based versus global approaches , 2003, Comput. Vis. Image Underst..

[31]  Hossein Mobahi,et al.  Toward a Practical Face Recognition System: Robust Alignment and Illumination by Sparse Representation , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Stefanos Zafeiriou,et al.  Subspace Learning from Image Gradient Orientations , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  David Zhang,et al.  Relaxed collaborative representation for pattern classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Cong Geng,et al.  Face recognition using sift features , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[35]  Hanspeter Pfister,et al.  Trainable Convolution Filters and Their Application to Face Recognition , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[37]  Tal Hassner,et al.  Effective Unconstrained Face Recognition by Combining Multiple Descriptors and Learned Background Statistics , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[39]  Yuan Yan Tang,et al.  Multiscale facial structure representation for face recognition under varying illumination , 2009, Pattern Recognit..

[40]  LinLin Shen,et al.  Local Gabor Binary Pattern Whitened PCA: A Novel Approach for Face Recognition from Single Image Per Person , 2009, ICB.

[41]  Yu Qiao,et al.  A Discriminative Feature Learning Approach for Deep Face Recognition , 2016, ECCV.

[42]  Xiaoyang Tan,et al.  Fusing Gabor and LBP Feature Sets for Kernel-Based Face Recognition , 2007, AMFG.

[43]  Shu Liao,et al.  Face recognition with salient local gradient orientation binary patterns , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[44]  Xudong Jiang,et al.  Sparse and Dense Hybrid Representation via Dictionary Decomposition for Face Recognition , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Tal Hassner,et al.  The One-Shot similarity kernel , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[46]  Gang Wang,et al.  Discriminative multi-manifold analysis for face recognition from a single training sample per person , 2011, ICCV.

[47]  Xiaoyang Tan,et al.  Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions , 2007, IEEE Transactions on Image Processing.

[48]  Javier Ruiz-del-Solar,et al.  A comparative study of thermal face recognition methods in unconstrained environments , 2012, Pattern Recognit..

[49]  Chengjun Liu,et al.  Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition , 2002, IEEE Trans. Image Process..

[50]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[51]  Jian Sun,et al.  Blessing of Dimensionality: High-Dimensional Feature and Its Efficient Compression for Face Verification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[52]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[53]  Oksam Chae,et al.  Local Directional Number Pattern for Face Analysis: Face and Expression Recognition , 2013, IEEE Transactions on Image Processing.

[54]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[55]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[56]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[57]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[58]  Nicolas Pinto,et al.  Establishing Good Benchmarks and Baselines for Face Recognition , 2008 .

[59]  Gang Wang,et al.  Discriminative multi-manifold analysis for face recognition from a single training sample per person , 2011, 2011 International Conference on Computer Vision.

[60]  Jun Guo,et al.  Extended SRC: Undersampled Face Recognition via Intraclass Variant Dictionary , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[61]  Simon C. K. Shiu,et al.  Gabor feature based robust representation and classification for face recognition with Gabor occlusion dictionary , 2013, Pattern Recognit..

[62]  Weilin Huang,et al.  Nonlinear Dimensionality Reduction for Face Recognition , 2009, IDEAL.

[63]  Qiang Ji,et al.  A Comparative Study of Local Matching Approach for Face Recognition , 2007, IEEE Transactions on Image Processing.

[64]  Shiv Ram Dubey,et al.  Rotation and Illumination Invariant Interleaved Intensity Order-Based Local Descriptor , 2014, IEEE Transactions on Image Processing.

[65]  Weilin Huang,et al.  Adaptive nonlinear manifolds and their applications to pattern recognition , 2010, Inf. Sci..

[66]  Yu-Chiang Frank Wang,et al.  Low-rank matrix recovery with structural incoherence for robust face recognition , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[67]  Marko Heikkilä,et al.  Description of interest regions with local binary patterns , 2009, Pattern Recognit..

[68]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[69]  Xiaogang Wang,et al.  Deeply learned face representations are sparse, selective, and robust , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[70]  Hyeonjoon Moon,et al.  The FERET evaluation methodology for face-recognition algorithms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[71]  Yuan Yan Tang,et al.  Face Recognition Under Varying Illumination Using Gradientfaces , 2009, IEEE Transactions on Image Processing.

[72]  Wen Gao,et al.  A comparative study on illumination preprocessing in face recognition , 2013, Pattern Recognit..

[73]  Sébastien Marcel,et al.  A Scalable Formulation of Probabilistic Linear Discriminant Analysis: Applied to Face Recognition , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[74]  Weilin Huang,et al.  Linear and nonlinear dimensionality reduction for face recognition , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[75]  Ran He,et al.  Maximum Correntropy Criterion for Robust Face Recognition , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[76]  Andrew Zisserman,et al.  Representing shape with a spatial pyramid kernel , 2007, CIVR '07.

[77]  Jun Guo,et al.  Robust, accurate and efficient face recognition from a single training image: A uniform pursuit approach , 2010, Pattern Recognit..

[78]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[79]  Ognjen Arandjelovic,et al.  Gradient Edge Map Features for Frontal Face Recognition under Extreme Illumination Changes , 2012, BMVC.

[80]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[81]  Baochang Zhang,et al.  Local Derivative Pattern Versus Local Binary Pattern: Face Recognition With High-Order Local Pattern Descriptor , 2010, IEEE Transactions on Image Processing.

[82]  Matthew A. Brown,et al.  Learning Local Image Descriptors , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[83]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[84]  James L. Crowley,et al.  Face Recognition using Tensors of Census Transform Histograms from Gaussian Features Maps , 2009, BMVC.

[85]  Alice Caplier,et al.  Enhanced Patterns of Oriented Edge Magnitudes for Face Recognition and Image Matching , 2012, IEEE Transactions on Image Processing.

[86]  Avinash C. Kak,et al.  PCA versus LDA , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[87]  B. S. Manjunath,et al.  Texture Features for Browsing and Retrieval of Image Data , 1996, IEEE Trans. Pattern Anal. Mach. Intell..