Chen et al . Face Quality Value : Input : Feat -‐ 5 Features : L 2 R + PKM Model : Feat -‐

Face image quality can be defined as a measure of the utility of a face image to automatic face recognition. In this work, we propose (and compare) two methods for learning face image quality based on target face quality values from (i) human assessments of face image quality (matcher-independent), and (ii) quality values computed from similarity scores (matcherdependent). A support vector regression model trained on face features extracted using a deep convolutional neural network (ConvNet) is used to predict the quality of a face image. The proposed methods are evaluated on two unconstrained face image databases, LFW and IJB-A, which both contain facial variations encompassing a multitude of quality factors. Evaluation of the proposed automatic face image quality measures shows we are able to reduce the FNMR at 1% FMR by at least 13% for two face matchers (a COTS matcher and a ConvNet matcher) by using the proposed face quality to select subsets of face images and video frames for matching templates (i.e., multiple faces per subject) in the IJB-A protocol. To our knowledge, this is the first work to utilize human assessments of face image quality in designing a predictor of unconstrained face quality that is shown to be effective in cross-database evaluation.

[1]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[2]  Patrick J. Flynn,et al.  Overview of the face recognition grand challenge , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[3]  Charles L. Wilson,et al.  A novel approach to fingerprint image quality , 2005, IEEE International Conference on Image Processing 2005.

[4]  B. Martin,et al.  Quality Assessment of Facial Images , 2006, 2006 Biometrics Symposium: Special Session on Research at the Biometric Consortium Conference.

[5]  Andy Adler,et al.  Human Vs. Automatic Measurement of Biometric Sample Quality , 2006, 2006 Canadian Conference on Electrical and Computer Engineering.

[6]  Elham Tabassi,et al.  Performance of Biometric Quality Measures , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[8]  Takeo Kanade,et al.  Multi-PIE , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[9]  Albert Ali Salah,et al.  Benchmarking Quality-Dependent and Cost-Sensitive Score-Level Multimodal Biometric Fusion Algorithms , 2009, IEEE Transactions on Information Forensics and Security.

[10]  Bruce A. Draper,et al.  Factors that influence algorithm performance in the Face Recognition Grand Challenge , 2009, Comput. Vis. Image Underst..

[11]  Julian Fiérrez,et al.  Quality-Based Conditional Processing in Multi-Biometrics: Application to Sensor Interoperability , 2010, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[12]  Bruce A. Draper,et al.  FRVT 2006: Quo Vadis face quality , 2010, Image Vis. Comput..

[13]  Sabah Jassim,et al.  Image-Quality-Based Adaptive Face Recognition , 2010, IEEE Transactions on Instrumentation and Measurement.

[14]  George W. Quinn,et al.  Report on the Evaluation of 2D Still-Image Face Recognition Algorithms , 2011 .

[15]  Bruce A. Draper,et al.  When high-quality face images match poorly , 2011, Face and Gesture 2011.

[16]  Patrick J. Flynn,et al.  Predicting performance of face recognition systems: An image characterization approach , 2011, CVPR 2011 WORKSHOPS.

[17]  Kristen Grauman,et al.  Relative attributes , 2011, 2011 International Conference on Computer Vision.

[18]  Yongkang Wong,et al.  Patch-based probabilistic image quality assessment for face selection and improved video-based face recognition , 2011, CVPR 2011 WORKSHOPS.

[19]  Bruce A. Draper,et al.  The Good, the Bad, and the Ugly Face Challenge Problem , 2012, Image and Vision Computing.

[20]  Anil K. Jain,et al.  Face Recognition Performance: Role of Demographic Information , 2012, IEEE Transactions on Information Forensics and Security.

[21]  Josef Kittler,et al.  A Unified Framework for Biometric Expert Fusion Incorporating Quality Measures , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Jian Sun,et al.  Blessing of Dimensionality: High-Dimensional Feature and Its Efficient Compression for Face Verification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Bruce A. Draper,et al.  On the existence of face quality measures , 2013, 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[24]  Samarth Bharadwaj,et al.  Can holistic representations be used for face biometric quality assessment? , 2013, 2013 IEEE International Conference on Image Processing.

[25]  Jinfeng Yi,et al.  Inferring Users' Preferences from Crowdsourced Pairwise Comparisons: A Matrix Completion Approach , 2013, HCOMP.

[26]  Richa Singh,et al.  MDLFace: Memorability augmented deep learning for video face recognition , 2014, IEEE International Joint Conference on Biometrics.

[27]  Abhishek Dutta,et al.  A Bayesian model for predicting face recognition performance using image quality , 2014, IEEE International Joint Conference on Biometrics.

[28]  Patrick J. Grother,et al.  Face Recognition Vendor Test (FRVT) Performance of Face Identification Algorithms NIST IR 8009 , 2014 .

[29]  Anil K. Jain,et al.  Unconstrained Face Recognition: Identifying a Person of Interest From a Media Collection , 2014, IEEE Transactions on Information Forensics and Security.

[30]  Shengcai Liao,et al.  A benchmark study of large-scale unconstrained face recognition , 2014, IEEE International Joint Conference on Biometrics.

[31]  Shengcai Liao,et al.  Learning Face Representation from Scratch , 2014, ArXiv.

[32]  Arun Ross,et al.  Design and evaluation of photometric image quality measures for effective face recognition , 2014, IET Biom..

[33]  Samarth Bharadwaj,et al.  Biometric quality: a review of fingerprint, iris, and face , 2014, EURASIP Journal on Image and Video Processing.

[34]  Sumohana S. Channappayya,et al.  Face image quality assessment for face selection in surveillance video using convolutional neural networks , 2015, 2015 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[35]  Matthew Q. Hill,et al.  Human and algorithm performance on the PaSC face Recognition Challenge , 2015, 2015 IEEE 7th International Conference on Biometrics Theory, Applications and Systems (BTAS).

[36]  Yong Man Ro,et al.  Face image assessment learned with objective and relative face image qualities for improved face recognition , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[37]  Yu Deng,et al.  Face Image Quality Assessment Based on Learning to Rank , 2015, IEEE Signal Processing Letters.

[38]  Anil K. Jain,et al.  Face Search at Scale: 80 Million Gallery , 2015, ArXiv.

[39]  Bruce A. Draper,et al.  Report on the FG 2015 Video Person Recognition Evaluation , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[40]  Anil K. Jain,et al.  Pushing the frontiers of unconstrained face detection and recognition: IARPA Janus Benchmark A , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Patrick J. Flynn,et al.  Report on the BTAS 2016 Video Person Recognition Evaluation , 2016, 2016 IEEE 8th International Conference on Biometrics Theory, Applications and Systems (BTAS).

[42]  Gérard G. Medioni,et al.  Pose-Aware Face Recognition in the Wild , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Tal Hassner,et al.  Do We Really Need to Collect Millions of Faces for Effective Face Recognition? , 2016, ECCV.

[44]  Carlos D. Castillo,et al.  Triplet probabilistic embedding for face verification and clustering , 2016, 2016 IEEE 8th International Conference on Biometrics Theory, Applications and Systems (BTAS).

[45]  Anil K. Jain,et al.  A Comparison of Human and Automated Face Verification Accuracy on Unconstrained Image Sets , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[46]  Rama Chellappa,et al.  Unconstrained face verification using deep CNN features , 2015, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[47]  Yiying Tong,et al.  Adaptive 3D Face Reconstruction from Unconstrained Photo Collections , 2016, CVPR.

[48]  Tal Hassner,et al.  Rapid Synthesis of Massive Face Sets for Improved Face Recognition , 2017, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[49]  Dongqing Zhang,et al.  Neural Aggregation Network for Video Face Recognition , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Richa Singh,et al.  Face Verification via Learned Representation on Feature-Rich Video Frames , 2017, IEEE Transactions on Information Forensics and Security.

[51]  Xiaoming Liu,et al.  Representation Learning by Rotating Your Faces , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.