Automatic Face Image Quality Prediction

Face image quality can be defined as a measure of the utility of a face image to automatic face recognition. In this work, we propose (and compare) two methods for automatic face image quality based on target face quality values from (i) human assessments of face image quality (matcher-independent), and (ii) quality values computed from similarity scores (matcher-dependent). A support vector regression model trained on face features extracted using a deep convolutional neural network (ConvNet) is used to predict the quality of a face image. The proposed methods are evaluated on two unconstrained face image databases, LFW and IJB-A, which both contain facial variations with multiple quality factors. Evaluation of the proposed automatic face image quality measures shows we are able to reduce the FNMR at 1% FMR by at least 13% for two face matchers (a COTS matcher and a ConvNet matcher) by using the proposed face quality to select subsets of face images and video frames for matching templates (i.e., multiple faces per subject) in the IJB-A protocol. To our knowledge, this is the first work to utilize human assessments of face image quality in designing a predictor of unconstrained face quality that is shown to be effective in cross-database evaluation.

[1]  Anil K. Jain,et al.  Pushing the frontiers of unconstrained face detection and recognition: IARPA Janus Benchmark A , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Anil K. Jain,et al.  Face Search at Scale: 80 Million Gallery , 2015, ArXiv.

[3]  Josef Kittler,et al.  A Unified Framework for Biometric Expert Fusion Incorporating Quality Measures , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Patrick J. Grother,et al.  Face Recognition Vendor Test (FRVT) Performance of Face Identification Algorithms NIST IR 8009 , 2014 .

[5]  B. Martin,et al.  Quality Assessment of Facial Images , 2006, 2006 Biometrics Symposium: Special Session on Research at the Biometric Consortium Conference.

[6]  Elham Tabassi,et al.  Performance of Biometric Quality Measures , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Samarth Bharadwaj,et al.  Can holistic representations be used for face biometric quality assessment? , 2013, 2013 IEEE International Conference on Image Processing.

[8]  Yiying Tong,et al.  Adaptive 3D Face Reconstruction from Unconstrained Photo Collections , 2016, CVPR.

[9]  Andy Adler,et al.  Human Vs. Automatic Measurement of Biometric Sample Quality , 2006, 2006 Canadian Conference on Electrical and Computer Engineering.

[10]  Arun Ross,et al.  Design and evaluation of photometric image quality measures for effective face recognition , 2014, IET Biom..

[11]  Bruce A. Draper,et al.  Factors that influence algorithm performance in the Face Recognition Grand Challenge , 2009, Comput. Vis. Image Underst..

[12]  Bruce A. Draper,et al.  The Good, the Bad, and the Ugly Face Challenge Problem , 2012, Image and Vision Computing.

[13]  Anil K. Jain,et al.  A Comparison of Human and Automated Face Verification Accuracy on Unconstrained Image Sets , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[14]  Shengcai Liao,et al.  Learning Face Representation from Scratch , 2014, ArXiv.

[15]  Jinfeng Yi,et al.  Inferring Users' Preferences from Crowdsourced Pairwise Comparisons: A Matrix Completion Approach , 2013, HCOMP.

[16]  Shengcai Liao,et al.  A benchmark study of large-scale unconstrained face recognition , 2014, IEEE International Joint Conference on Biometrics.

[17]  Yong Man Ro,et al.  Face image assessment learned with objective and relative face image qualities for improved face recognition , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[18]  Abhishek Dutta,et al.  A Bayesian model for predicting face recognition performance using image quality , 2014, IEEE International Joint Conference on Biometrics.

[19]  Yongkang Wong,et al.  Patch-based probabilistic image quality assessment for face selection and improved video-based face recognition , 2011, CVPR 2011 WORKSHOPS.

[20]  Anil K. Jain,et al.  Unconstrained Face Recognition: Identifying a Person of Interest From a Media Collection , 2014, IEEE Transactions on Information Forensics and Security.

[21]  Julian Fiérrez,et al.  Quality Measures in Biometric Systems , 2012, IEEE Security & Privacy.

[22]  Charles L. Wilson,et al.  A novel approach to fingerprint image quality , 2005, IEEE International Conference on Image Processing 2005.

[23]  Sabah Jassim,et al.  Image-Quality-Based Adaptive Face Recognition , 2010, IEEE Transactions on Instrumentation and Measurement.

[24]  Xiaoming Liu,et al.  Disentangled Representation Learning GAN for Pose-Invariant Face Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Bruce A. Draper,et al.  FRVT 2006: Quo Vadis face quality , 2010, Image Vis. Comput..

[26]  Samarth Bharadwaj,et al.  Biometric quality: a review of fingerprint, iris, and face , 2014, EURASIP Journal on Image and Video Processing.

[27]  Rama Chellappa,et al.  Unconstrained face verification using deep CNN features , 2015, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[28]  Anil K. Jain,et al.  Face Recognition Performance: Role of Demographic Information , 2012, IEEE Transactions on Information Forensics and Security.

[29]  Kristen Grauman,et al.  Relative attributes , 2011, 2011 International Conference on Computer Vision.

[30]  Bruce A. Draper,et al.  On the existence of face quality measures , 2013, 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[31]  George W. Quinn,et al.  Report on the Evaluation of 2D Still-Image Face Recognition Algorithms , 2011 .

[32]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[33]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[34]  Takeo Kanade,et al.  Multi-PIE , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[35]  Yu Deng,et al.  Face Image Quality Assessment Based on Learning to Rank , 2015, IEEE Signal Processing Letters.

[36]  Jian Sun,et al.  Blessing of Dimensionality: High-Dimensional Feature and Its Efficient Compression for Face Verification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Patrick J. Flynn,et al.  Predicting performance of face recognition systems: An image characterization approach , 2011, CVPR 2011 WORKSHOPS.

[38]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[39]  Patrick J. Flynn,et al.  Overview of the face recognition grand challenge , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).