Learning Face Image Quality From Human Assessments

Face image quality can be defined as a measure of the utility of a face image to automatic face recognition. In this paper, we propose (and compare) two methods for learning face image quality based on target face quality values from: 1) human assessments of face image quality (matcher-independent) and 2) quality values computed from similarity scores (matcher-dependent). A support vector regression model trained on face features extracted using a deep convolutional neural network (ConvNet) is used to predict the quality of a face image. The proposed methods are evaluated on two unconstrained face image databases, Labeled Faces in the Wild and IARPA Janus Benchmark-A (IJB-A), which both contain facial variations encompassing a multitude of quality factors. Evaluation of the proposed automatic face image quality measures shows we are able to reduce the false non-match rate at 1% false match rate by at least 13% for two face matchers (a commercial off-the-shelf matcher and a ConvNet matcher) by using the proposed face quality to select subsets of face images and video frames for matching templates (i.e., multiple faces per subject) in the IJB-A protocol. To the best of our knowledge, this is the first work to utilize human assessments of face image quality in designing a predictor of unconstrained face quality that is shown to be effective in cross-database evaluation.

[1]  Jian Sun,et al.  Blessing of Dimensionality: High-Dimensional Feature and Its Efficient Compression for Face Verification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Gérard G. Medioni,et al.  Pose-Aware Face Recognition in the Wild , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Bruce A. Draper,et al.  FRVT 2006: Quo Vadis face quality , 2010, Image Vis. Comput..

[4]  Arun Ross,et al.  Design and evaluation of photometric image quality measures for effective face recognition , 2014, IET Biom..

[5]  Patrick J. Flynn,et al.  Predicting performance of face recognition systems: An image characterization approach , 2011, CVPR 2011 WORKSHOPS.

[6]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[7]  Dongqing Zhang,et al.  Neural Aggregation Network for Video Face Recognition , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Bruce A. Draper,et al.  Report on the FG 2015 Video Person Recognition Evaluation , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[9]  Takeo Kanade,et al.  Multi-PIE , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[10]  Patrick J. Flynn,et al.  Overview of the face recognition grand challenge , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  Patrick J. Grother,et al.  Face Recognition Vendor Test (FRVT) Performance of Face Identification Algorithms NIST IR 8009 , 2014 .

[12]  Tal Hassner,et al.  Rapid Synthesis of Massive Face Sets for Improved Face Recognition , 2017, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[13]  Josef Kittler,et al.  A Unified Framework for Biometric Expert Fusion Incorporating Quality Measures , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[15]  Shengcai Liao,et al.  A benchmark study of large-scale unconstrained face recognition , 2014, IEEE International Joint Conference on Biometrics.

[16]  Rama Chellappa,et al.  Unconstrained face verification using deep CNN features , 2015, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[17]  Anil K. Jain,et al.  Face Recognition Performance: Role of Demographic Information , 2012, IEEE Transactions on Information Forensics and Security.

[18]  George W. Quinn,et al.  Report on the Evaluation of 2D Still-Image Face Recognition Algorithms , 2011 .

[19]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[20]  Kristen Grauman,et al.  Relative attributes , 2011, 2011 International Conference on Computer Vision.

[21]  Andy Adler,et al.  Human Vs. Automatic Measurement of Biometric Sample Quality , 2006, 2006 Canadian Conference on Electrical and Computer Engineering.

[22]  Anil K. Jain,et al.  Unconstrained Face Recognition: Identifying a Person of Interest From a Media Collection , 2014, IEEE Transactions on Information Forensics and Security.

[23]  Bruce A. Draper,et al.  When high-quality face images match poorly , 2011, Face and Gesture 2011.

[24]  Julian Fiérrez,et al.  Quality Measures in Biometric Systems , 2012, IEEE Security & Privacy.

[25]  Yu Deng,et al.  Face Image Quality Assessment Based on Learning to Rank , 2015, IEEE Signal Processing Letters.

[26]  Julian Fiérrez,et al.  Quality-Based Conditional Processing in Multi-Biometrics: Application to Sensor Interoperability , 2010, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[27]  Samarth Bharadwaj,et al.  Can holistic representations be used for face biometric quality assessment? , 2013, 2013 IEEE International Conference on Image Processing.

[28]  Bruce A. Draper,et al.  Factors that influence algorithm performance in the Face Recognition Grand Challenge , 2009, Comput. Vis. Image Underst..

[29]  Bruce A. Draper,et al.  The Good, the Bad, and the Ugly Face Challenge Problem , 2012, Image and Vision Computing.

[30]  Samarth Bharadwaj,et al.  Biometric quality: a review of fingerprint, iris, and face , 2014, EURASIP Journal on Image and Video Processing.

[31]  Richa Singh,et al.  MDLFace: Memorability augmented deep learning for video face recognition , 2014, IEEE International Joint Conference on Biometrics.

[32]  Sumohana S. Channappayya,et al.  Face image quality assessment for face selection in surveillance video using convolutional neural networks , 2015, 2015 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[33]  Yiying Tong,et al.  Adaptive 3D Face Reconstruction from Unconstrained Photo Collections , 2016, CVPR.

[34]  Abhishek Dutta,et al.  A Bayesian model for predicting face recognition performance using image quality , 2014, IEEE International Joint Conference on Biometrics.

[35]  Patrick J. Flynn,et al.  Report on the BTAS 2016 Video Person Recognition Evaluation , 2016, 2016 IEEE 8th International Conference on Biometrics Theory, Applications and Systems (BTAS).

[36]  Anil K. Jain,et al.  A Comparison of Human and Automated Face Verification Accuracy on Unconstrained Image Sets , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[37]  Shengcai Liao,et al.  Learning Face Representation from Scratch , 2014, ArXiv.

[38]  Yong Man Ro,et al.  Face image assessment learned with objective and relative face image qualities for improved face recognition , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[39]  Elham Tabassi,et al.  Performance of Biometric Quality Measures , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Tal Hassner,et al.  Do We Really Need to Collect Millions of Faces for Effective Face Recognition? , 2016, ECCV.

[41]  Richa Singh,et al.  Face Verification via Learned Representation on Feature-Rich Video Frames , 2017, IEEE Transactions on Information Forensics and Security.

[42]  Yongkang Wong,et al.  Patch-based probabilistic image quality assessment for face selection and improved video-based face recognition , 2011, CVPR 2011 WORKSHOPS.

[43]  Matthew Q. Hill,et al.  Human and algorithm performance on the PaSC face Recognition Challenge , 2015, 2015 IEEE 7th International Conference on Biometrics Theory, Applications and Systems (BTAS).

[44]  B. Martin,et al.  Quality Assessment of Facial Images , 2006, 2006 Biometrics Symposium: Special Session on Research at the Biometric Consortium Conference.

[45]  Charles L. Wilson,et al.  A novel approach to fingerprint image quality , 2005, IEEE International Conference on Image Processing 2005.

[46]  Bruce A. Draper,et al.  On the existence of face quality measures , 2013, 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[47]  Carlos D. Castillo,et al.  Triplet probabilistic embedding for face verification and clustering , 2016, 2016 IEEE 8th International Conference on Biometrics Theory, Applications and Systems (BTAS).

[48]  Jinfeng Yi,et al.  Inferring Users' Preferences from Crowdsourced Pairwise Comparisons: A Matrix Completion Approach , 2013, HCOMP.

[49]  Anil K. Jain,et al.  Face Search at Scale , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Xiaoming Liu,et al.  Representation Learning by Rotating Your Faces , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  Sabah Jassim,et al.  Image-Quality-Based Adaptive Face Recognition , 2010, IEEE Transactions on Instrumentation and Measurement.

[52]  Albert Ali Salah,et al.  Benchmarking Quality-Dependent and Cost-Sensitive Score-Level Multimodal Biometric Fusion Algorithms , 2009, IEEE Transactions on Information Forensics and Security.

[53]  Anil K. Jain,et al.  Pushing the frontiers of unconstrained face detection and recognition: IARPA Janus Benchmark A , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).