Face retriever: Pre-filtering the gallery via deep neural net

Face retrieval is an enabling technology for many applications, including automatic face annotation, deduplication, and surveillance. In this paper, we propose a face retrieval system which combines a k-NN search procedure with a COTS matcher (PittPatt1) in a cascaded manner. In particular, given a query face, we first pre-filter the gallery set and find the top-k most similar faces for the query image by using deep facial features that are learned with a deep convolutional neural network. The top-k most similar faces are then re-ranked based on score-level fusion of the similarities between deep features and the COTS matcher. To further boost the retrieval performance, we develop a manifold ranking algorithm. The proposed face retrieval system is evaluated on two large-scale face image databases: (i) a web face image database, which consists of over 3, 880 query images of 1, 507 subjects and a gallery of 5, 000, 000 faces, and (ii) a mugshot database, which consists of 1, 000 query images of 1, 000 subjects and a gallery of 1, 000, 000 faces. Experimental results demonstrate that the proposed face retrieval system can simultaneously improve the retrieval performance (CMC and precision-recall) and scalability for large-scale face retrieval problems.

[1]  Xiaogang Wang,et al.  Deep Learning Face Representation by Joint Identification-Verification , 2014, NIPS.

[2]  Ji Wan,et al.  Deep Learning for Content-Based Image Retrieval: A Comprehensive Study , 2014, ACM Multimedia.

[3]  Yan-Ying Chen,et al.  Scalable Face Image Retrieval Using Attribute-Enhanced Sparse Codewords , 2013, IEEE Transactions on Multimedia.

[4]  Ying He,et al.  Retrieval-Based Face Annotation by Weak Label Regularized Local Coordinate Coding , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[6]  Qingshan Liu,et al.  Image retrieval via probabilistic hypergraph ranking , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Enrique G. Ortiz,et al.  Evaluating Open-Universe Face Identification on the Web , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[8]  Anil K. Jain,et al.  Handbook of Face Recognition, 2nd Edition , 2011 .

[9]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[10]  Xiaogang Wang,et al.  Hybrid Deep Learning for Face Verification , 2013, 2013 IEEE International Conference on Computer Vision.

[11]  Zhanguo Wei,et al.  Image Reranking by Example : A Semi-supervised Learning Formulation , 2010 .

[12]  Zhe Wang,et al.  Modeling LSH for performance tuning , 2008, CIKM '08.

[13]  Xiaogang Wang,et al.  Deep Learning Face Representation from Predicting 10,000 Classes , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Harry Shum,et al.  Scalable face image retrieval with identity-based quantization and multi-reference re-ranking , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Trevor Darrell,et al.  DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition , 2013, ICML.