Partial Face Recognition: Alignment-Free Approach

Numerous methods have been developed for holistic face recognition with impressive performance. However, few studies have tackled how to recognize an arbitrary patch of a face image. Partial faces frequently appear in unconstrained scenarios, with images captured by surveillance cameras or handheld devices (e.g., mobile phones) in particular. In this paper, we propose a general partial face recognition approach that does not require face alignment by eye coordinates or any other fiducial points. We develop an alignment-free face representation method based on Multi-Keypoint Descriptors (MKD), where the descriptor size of a face is determined by the actual content of the image. In this way, any probe face image, holistic or partial, can be sparsely represented by a large dictionary of gallery descriptors. A new keypoint descriptor called Gabor Ternary Pattern (GTP) is also developed for robust and discriminative face recognition. Experimental results are reported on four public domain face databases (FRGCv2.0, AR, LFW, and PubFig) under both the open-set identification and verification scenarios. Comparisons with two leading commercial face recognition SDKs (PittPatt and FaceVACS) and two baseline algorithms (PCA+LDA and LBP) show that the proposed method, overall, is superior in recognizing both holistic and partial faces without requiring alignment.

[1]  Hongjun Jia,et al.  Support Vector Machines in face recognition with occlusions , 2009, CVPR.

[2]  Enrico Grosso,et al.  Face Identification by SIFT-based Complete Graph Topology , 2007, 2007 IEEE Workshop on Automatic Identification Advanced Technologies.

[3]  Chi-Ho Chan,et al.  Sparse representation of (Multiscale) histograms for face recognition robust to registration and illumination problems , 2010, 2010 IEEE International Conference on Image Processing.

[4]  David Beymer,et al.  Face recognition under varying pose , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Yaakov Tsaig,et al.  Fast Solution of $\ell _{1}$ -Norm Minimization Problems When the Solution May Be Sparse , 2008, IEEE Transactions on Information Theory.

[6]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Jian Sun,et al.  An associate-predict model for face recognition , 2011, CVPR 2011.

[8]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[9]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[10]  Patrick J. Flynn,et al.  Overview of the face recognition grand challenge , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  Shengcai Liao,et al.  Part-based Face Recognition Using Near Infrared Images , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  David R. Musser.  Introspective Sorting and Selection Algorithms , 1997 .

[13]  Tomaso A. Poggio,et al.  Face recognition: component-based versus global approaches , 2003, Comput. Vis. Image Underst..

[14]  Nicolas Pinto,et al.  How far can you get with a modern face recognition test set using only simple features? , 2009, CVPR.

[15]  A. Martínez,et al.  The AR face databasae , 1998 .

[16]  Jongsun Kim,et al.  Effective representation using ICA for face recognition robust to local distortion and partial occlusion , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Kazuhiro Hotta.  Robust face recognition under partial occlusion based on support vector machine with local Gaussian summation kernel , 2008, Image Vis. Comput..

[18]  Dong-mei Sun,et al.  Bag-of-Words Vector Quantization Based Face Identification , 2009, 2009 Second International Symposium on Electronic Commerce and Security.

[19]  Gang Hua,et al.  Introduction to the Special Section on Real-World Face Recognition , 2011, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Shree K. Nayar,et al.  Attribute and simile classifiers for face verification , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[21]  Xiaoyang Tan,et al.  Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions , 2007, AMFG.

[22]  Anil K. Jain,et al.  Periocular biometrics in the visible spectrum: A feasibility study , 2009, 2009 IEEE 3rd International Conference on Biometrics: Theory, Applications, and Systems.

[23]  Shie Mannor,et al.  Sparse Algorithms Are Not Stable: A No-Free-Lunch Theorem , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Alex Pentland,et al.  View-based and modular eigenspaces for face recognition , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Tomaso A. Poggio,et al.  Linear Object Classes and Image Synthesis From a Single Example Image , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Harry Wechsler,et al.  The FERET database and evaluation procedure for face-recognition algorithms , 1998, Image Vis. Comput..

[27]  Ralph Gross,et al.  Fisher Light-Fields for Face Recognition across Pose and Illumination , 2002, DAGM-Symposium.

[28]  Sang Uk Lee,et al.  Occlusion invariant face recognition using selective local non-negative matrix factorization basis images , 2008, Image Vis. Comput..

[29]  Kuldip K. Paliwal,et al.  Polynomial features for robust face authentication , 2002, Proceedings. International Conference on Image Processing.

[30]  Sami Romdhani,et al.  Face identification across different poses and illuminations with a 3D morphable model , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[31]  Mohammed Bennamoun,et al.  An Efficient Multimodal 2D-3D Hybrid Approach to Automatic Face Recognition , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Aleix M. Martínez,et al.  Recognizing Imprecisely Localized, Partially Occluded, and Expression Variant Faces from a Single Sample per Class , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Tsuhan Chen,et al.  A GMM parts based face representation for improved verification through relevance adaptation , 2004, CVPR 2004.

[34]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Roberto Brunelli,et al.  Face Recognition: Features Versus Templates , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Sébastien Marcel,et al.  Comparison of MLP and GMM Classifiers for Face Verification on XM2VTS , 2003, AVBPA.

[37]  Andrea Lagorio,et al.  On the Use of SIFT Features for Face Authentication , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[38]  Chengjun Liu,et al.  Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition , 2002, IEEE Trans. Image Process..

[39]  Hossein Mobahi,et al.  Toward a Practical Face Recognition System: Robust Alignment and Illumination by Sparse Representation , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Samy Bengio,et al.  User authentication via adapted statistical models of face images , 2006, IEEE Transactions on Signal Processing.

[41]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[42]  Cordelia Schmid,et al.  Shape recognition with edge-based features , 2003, BMVC.

[43]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[44]  Timothy F. Cootes,et al.  Active Appearance Models , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[45]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[46]  Rainer Stiefelhagen,et al.  Why Is Facial Occlusion a Challenging Problem? , 2009, ICB.

[47]  Jake K. Aggarwal,et al.  Partial face recognition using radial basis function networks , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[48]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[49]  Shengcai Liao,et al.  Partial Face Matching between Near Infrared and Visual Images in MBGC Portal Challenge , 2009, ICB.

[50]  Shengcai Liao,et al.  Automatic Partial Face Alignment in NIR Video Sequences , 2009, ICB.

[51]  Tony Lindeberg,et al.  Feature Detection with Automatic Scale Selection , 1998, International Journal of Computer Vision.

[52]  Abdenour Hadid,et al.  Improving the recognition of faces occluded by facial accessories , 2011, Face and Gesture 2011.

[53]  Norbert Krüger,et al.  Face recognition by elastic bunch graph matching , 1997, Proceedings of International Conference on Image Processing.

[54]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[55]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  Masahide Kaneko,et al.  Robust Face Recognition Using Block-Based Bag of Words , 2010, 2010 20th International Conference on Pattern Recognition.

[57]  Zihan Zhou,et al.  Towards a practical face recognition system: Robust registration and illumination by sparse representation , 2009, CVPR.

[58]  Cordelia Schmid,et al.  An Affine Invariant Interest Point Detector , 2002, ECCV.

[59]  Lei Zhang,et al.  Sparse representation or collaborative representation: Which helps face recognition? , 2011, 2011 International Conference on Computer Vision.

[60]  Stan Z. Li,et al.  Learning spatially localized, parts-based representation , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[61]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[62]  D. B. Graham,et al.  Face recognition from unfamiliar views: subspace methods and pose dependency , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[63]  David Beymer,et al.  Face recognition from one example view , 1995, Proceedings of IEEE International Conference on Computer Vision.

[64]  Cordelia Schmid,et al.  Scale & Affine Invariant Interest Point Detectors , 2004, International Journal of Computer Vision.

[65]  Jun Luo,et al.  Person-Specific SIFT Features for Face Recognition , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[66]  Norbert Krüger,et al.  Face Recognition by Elastic Bunch Graph Matching , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[67]  Srinivas Gutta,et al.  An investigation into the use of partial-faces for face recognition , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.