EPML: Expanded Parts Based Metric Learning for Occlusion Robust Face Verification

We propose a novel Expanded Parts based Metric Learning (EPML) model for face verification. The model is capable of mining out the discriminative regions at the right locations and scales, for identity based matching of face images. It performs well in the presence of occlusions, by avoiding the occluded regions and selecting the next best visible regions. We show quantitatively, by experiments on the standard benchmark dataset Labeled Faces in the Wild (LFW), that the model works much better than the traditional method of face representation with metric learning, both (i) in the presence of heavy random occlusions and, (ii) also, in the case of focussed occlusions of discriminative face regions such as eyes or mouth. Further, we present qualitative results which demonstrate that the method is capable of ignoring the occluded regions while exploiting the visible ones.

[1]  Deyu Meng,et al.  Towards Efficient Learning of Optimal Spatial Bag-of-Words Representations , 2014, ICMR.

[2]  Cordelia Schmid,et al.  Is that you? Metric learning approaches for face identification , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[3]  Peter N. Belhumeur,et al.  Tom-vs-Pete Classifiers and Identity-Preserving Alignment for Face Verification , 2012, BMVC.

[4]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[5]  Li Bai,et al.  Cosine Similarity Metric Learning for Face Verification , 2010, ACCV.

[6]  Jiwen Lu,et al.  Robust Feature Set Matching for Partial Face Recognition , 2013, 2013 IEEE International Conference on Computer Vision.

[7]  Andrew Zisserman,et al.  Fisher Vector Faces in the Wild , 2013, BMVC.

[8]  Shengcai Liao,et al.  Partial Face Recognition: Alignment-Free Approach , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Yutaka Satoh,et al.  Robust face recognition using the GAP feature , 2013, Pattern Recognit..

[10]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Frédéric Jurie,et al.  PCCA: A new approach for distance learning from sparse pairwise constraints , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Gang Hua,et al.  Probabilistic Elastic Matching for Pose Variant Face Verification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Cordelia Schmid,et al.  Expanded Parts Model for Human Attribute and Action Recognition in Still Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Julio Jacobo-Berlles,et al.  Face recognition on partially occluded images using compressed sensing , 2014, Pattern Recognit. Lett..

[15]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Berk Gökberk,et al.  3-D Face Recognition Under Occlusion Using Masked Projection , 2013, IEEE Transactions on Information Forensics and Security.

[17]  Xiaogang Wang,et al.  Hybrid Deep Learning for Face Verification , 2013, 2013 IEEE International Conference on Computer Vision.

[18]  Oren Barkan,et al.  Fast High Dimensional Vector Multiplication Face Recognition , 2013, 2013 IEEE International Conference on Computer Vision.

[19]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[21]  Shiguang Shan,et al.  Fusing Robust Face Region Descriptors via Multiple Metric Learning for Face Recognition in the Wild , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[23]  Horst Bischof,et al.  Dealing with occlusions in the eigenspace approach , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Shree K. Nayar,et al.  Attribute and simile classifiers for face verification , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[25]  Raimondo Schettini,et al.  Three-Dimensional Occlusion Detection and Restoration of Partially Occluded Faces , 2011, Journal of Mathematical Imaging and Vision.

[26]  Abdenour Hadid,et al.  Efficient Detection of Occlusion prior to Robust Face Recognition , 2014, TheScientificWorldJournal.

[27]  Gaurav Sharma,et al.  Learning discriminative spatial representation for image classification , 2011, BMVC.

[28]  Dahua Lin,et al.  Quality-Driven Face Occlusion Detection and Recovery , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Abdenour Hadid,et al.  Improving the recognition of faces occluded by facial accessories , 2011, Face and Gesture 2011.

[30]  Thomas Sikora,et al.  More robust face recognition by considering occlusion information , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[31]  Peng Li,et al.  Similarity Metric Learning for Face Recognition , 2013, 2013 IEEE International Conference on Computer Vision.

[32]  Sang Uk Lee,et al.  Occlusion invariant face recognition using selective local non-negative matrix factorization basis images , 2008, Image Vis. Comput..

[33]  R. Schettini,et al.  Recognizing Faces In 3D Images Even In Presence Of Occlusions , 2008, 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems.

[34]  Andrew Zisserman,et al.  Deep Fisher Networks for Large-Scale Image Classification , 2013, NIPS.

[35]  Xinge You,et al.  Robust face recognition via occlusion dictionary learning , 2014, Pattern Recognit..

[36]  Jian Sun,et al.  Blessing of Dimensionality: High-Dimensional Feature and Its Efficient Compression for Face Verification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[38]  Lawrence Sirovich,et al.  Karhunen–Loève procedure for gappy data , 1995 .

[39]  Alexei A. Efros,et al.  Mid-level Visual Element Discovery as Discriminative Mode Seeking , 2013, NIPS.

[40]  Tal Hassner,et al.  Similarity Scores Based on Background Samples , 2009, ACCV.

[41]  Raimondo Schettini,et al.  Detection and Restoration of Occlusions for 3D Face Recognition , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[42]  Hossein Mobahi,et al.  Face recognition with contiguous occlusion using markov random fields , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[43]  Peter N. Belhumeur,et al.  POOF: Part-Based One-vs.-One Features for Fine-Grained Categorization, Face Verification, and Attribute Estimation , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[44]  Koichi Ito,et al.  Performance improvement of face recognition algorithms using occluded-region detection , 2013, 2013 International Conference on Biometrics (ICB).