The Effect of Wearing a Face Mask on Face Image Quality

Due to the COVID-19 situation, face masks have become a main part of our daily life. Wearing mouth-and-nose protection has been made a mandate in many public places, to prevent the spread of the COVID-19 virus. However, face masks affect the performance of face recognition, since a large area of the face is covered. The effect of wearing a face mask on the different components of the face recognition system in a collaborative environment is a problem that is still to be fully studied. This work studies, for the first time, the effect of wearing a face mask on face image quality by utilising state-of-the-art face image quality assessment methods of different natures. This aims at providing better understanding on the effect of face masks on the operation of face recognition as a whole system. In addition, we further studied the effect of simulated masks on face image utility in comparison to real face masks. We discuss the correlation between the mask effect on face image quality and that on the face verification performance by automatic systems and human experts, indicating a consistent trend between both factors. The evaluation is conducted on the database containing (1) no-masked faces, (2) real face masks, and (3) simulated face masks, by synthetically generating digital facial masks on no-masked faces. Finally, a visual interpretation of the face areas contributing to the quality score of a selected set of quality assessment methods is provided to give a deeper insight into the difference of network decisions in masked and non-masked faces, among other variations.

[1]  Naser Damer,et al.  Real Masks and Fake Faces: On the Masked Face Presentation Attack Detection , 2021, ArXiv.

[2]  Yu Deng,et al.  Face Image Quality Assessment Based on Learning to Rank , 2015, IEEE Signal Processing Letters.

[3]  Arjan Kuijper,et al.  Extended evaluation of the effect of real and simulated masks on face recognition performance , 2021, IET Biom..

[4]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[5]  Yuxiao Hu,et al.  MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition , 2016, ECCV.

[6]  Christoph Busch,et al.  Social acceptance of biometric technologies in Germany: A survey , 2013, 2013 International Conference of the BIOSIG Special Interest Group (BIOSIG).

[7]  S. Zafeiriou,et al.  ArcFace: Additive Angular Margin Loss for Deep Face Recognition. , 2021, IEEE transactions on pattern analysis and machine intelligence.

[8]  Shichao Zhao,et al.  MagFace: A Universal Representation for Face Recognition and Quality Assessment , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Stefanos Zafeiriou,et al.  ArcFace: Additive Angular Margin Loss for Deep Face Recognition , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Bhiksha Raj,et al.  SphereFace: Deep Hypersphere Embedding for Face Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Naser Damer,et al.  A Deep Insight into Measuring Face Image Utility with General and Face-specific Image Quality Metrics , 2021, 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV).

[12]  Patrick Grother,et al.  Ongoing Face Recognition Vendor Test (FRVT) Part 6B: Face recognition accuracy with face masks using post-COVID-19 algorithms , 2020 .

[13]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[14]  Naser Damer,et al.  ElasticFace: Elastic Margin Loss for Deep Face Recognition , 2021, ArXiv.

[15]  Naser Damer,et al.  Biometric source weighting in multi-biometric fusion: Towards a generalized and robust solution , 2014, 2014 22nd European Signal Processing Conference (EUSIPCO).

[16]  Javier Hernandez-Ortega,et al.  FaceQnet: Quality Assessment for Face Recognition based on Deep Learning , 2019, 2019 International Conference on Biometrics (ICB).

[17]  Naser Damer,et al.  Unmasking Face Embeddings by Self-restrained Triplet Loss for Accurate Masked Face Recognition , 2021, Pattern Recognition.

[18]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Patrick J. Grother,et al.  Ongoing Face Recognition Vendor Test (FRVT) Part 2: Identification , 2018 .

[20]  Naser Damer,et al.  My Eyes Are Up Here: Promoting Focus on Uncovered Regions in Masked Face Recognition , 2021, 2021 International Conference of the Biometrics Special Interest Group (BIOSIG).

[21]  Naser Damer,et al.  PocketNet: Extreme Lightweight Face Recognition Network using Neural Architecture Search and Multi-Step Knowledge Distillation , 2021, IEEE Access.

[22]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Davis E. King,et al.  Dlib-ml: A Machine Learning Toolkit , 2009, J. Mach. Learn. Res..

[24]  Naser Damer,et al.  The Effect of Wearing a Mask on Face Recognition Performance: an Exploratory Study , 2020, 2020 International Conference of the Biometrics Special Interest Group (BIOSIG).

[25]  Christoph Busch,et al.  Face Image Quality Assessment: A Literature Survey , 2020, ACM Comput. Surv..

[26]  Gengming Zhu,et al.  Joint Face Detection and Facial Expression Recognition with MTCNN , 2017, 2017 4th International Conference on Information Science and Control Engineering (ICISCE).

[27]  Naser Damer,et al.  SER-FIQ: Unsupervised Estimation of Face Image Quality Based on Stochastic Embedding Robustness , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Yu Qiao,et al.  Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks , 2016, IEEE Signal Processing Letters.

[29]  Zijian Zhang,et al.  Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[30]  Julian Fierrez,et al.  Biometric Quality: Review and Application to Face Recognition with FaceQnet , 2020, ArXiv.

[31]  Naser Damer,et al.  Biometrics in the Era of COVID-19: Challenges and Opportunities , 2021, ArXiv.

[32]  Shiguang Shan,et al.  MFR 2021: Masked Face Recognition Competition , 2021, 2021 IEEE International Joint Conference on Biometrics (IJCB).

[33]  Andrew Zisserman,et al.  Deep Face Recognition , 2015, BMVC.