Boosting Cross-Quality Face Verification Using Blind Face Restoration

In recent years, various Blind Face Restoration (BFR) techniques were developed. These techniques transform low quality faces suffering from multiple degradations to more realistic and natural face images with high perceptual quality. However, it is crucial for the task of face verification to not only enhance the perceptual quality of the low quality images but also to improve the biometric-utility face quality metrics. Furthermore, preserving the valuable identity information is of great importance. In this paper, we investigate the impact of applying three state-of-the-art blind face restoration techniques namely, GFP-GAN, GPEN and SGPN on the performance of face verification system under very challenging environment characterized by very low quality images. Extensive experimental results on the recently proposed cross-quality LFW database using three state-of-the-art deep face recognition models demonstrate the effectiveness of GFP-GAN in boosting significantly the face verification accuracy.

[1]  Xiaochun Cao,et al.  A Survey of Deep Face Restoration: Denoise, Super-Resolution, Deblur, Artifact Removal , 2022, ArXiv.

[2]  Jaime S. Cardoso,et al.  OCFR 2022: Competition on Occluded Face Recognition from Synthetically Generated Structure-Aware Occlusions , 2022, 2022 IEEE International Joint Conference on Biometrics (IJCB).

[3]  Kaihao Zhang,et al.  Blind Face Restoration: Benchmark Datasets and a Baseline Model , 2022, Neurocomputing.

[4]  Ying Tai,et al.  Blind Face Restoration via Integrating Face Shape and Generative Priors , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Dalong Du,et al.  WebFace260M: A Benchmark for Million-Scale Deep Face Recognition , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Anil K. Jain,et al.  AdaFace: Quality Adaptive Margin for Face Recognition , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  P. Luo,et al.  RestoreFormer: High-Quality Blind Face Restoration from Undegraded Key-Value Pairs , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  N. Damer,et al.  CR-FIQA: Face Image Quality Assessment by Learning Sample Relative Classifiability , 2021, 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Bjoern H Menze,et al.  Face Restoration via Plug-and-Play 3D Facial Priors , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Inad A. Aljarrah,et al.  Effect of image degradation on performance of Convolutional Neural Networks , 2021, Int. J. Commun. Networks Inf. Secur..

[11]  G. Rigoll,et al.  Cross-Quality LFW: A Database for Analyzing Cross- Resolution Image Face Recognition in Unconstrained Environments , 2021, 2021 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021).

[12]  Peiran Ren,et al.  GAN Prior Embedded Network for Blind Face Restoration in the Wild , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  F. Zhou,et al.  MagFace: A Universal Representation for Face Recognition and Quality Assessment , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Shiqi Wang,et al.  Comparison of Full-Reference Image Quality Models for Optimization of Image Processing Systems , 2021, Int. J. Comput. Vis..

[15]  Xintao Wang,et al.  Towards Real-World Blind Face Restoration with Generative Facial Prior , 2021, Computer Vision and Pattern Recognition.

[16]  Kwan-Yee Kenneth Wong,et al.  Learning Spatial Attention for Face Super-Resolution , 2020, IEEE Transactions on Image Processing.

[17]  Tao Mei,et al.  The Elements of End-to-end Deep Face Recognition: A Survey of Recent Advances , 2020, ACM Comput. Surv..

[18]  Lei Zhang,et al.  Progressive Semantic-Aware Style Transformation for Blind Face Restoration , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Christoph Busch,et al.  Face Image Quality Assessment: A Literature Survey , 2020, ACM Comput. Surv..

[20]  Meng Wang,et al.  Enhanced Blind Face Restoration With Multi-Exemplar Images and Adaptive Spatial Feature Fusion , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  W. Gao,et al.  HiFaceGAN: Face Renovation via Collaborative Suppression and Replenishment , 2020, ACM Multimedia.

[22]  Wangmeng Zuo,et al.  Attention-guided CNN for image denoising , 2020, Neural Networks.

[23]  Tero Karras,et al.  Analyzing and Improving the Image Quality of StyleGAN , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Timo Aila,et al.  A Style-Based Generator Architecture for Generative Adversarial Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Patrick J. Flynn,et al.  Face Recognition in Low Quality Images: A Survey , 2018, ArXiv.

[26]  Mei Wang,et al.  Deep Face Recognition: A Survey , 2018, Neurocomputing.

[27]  Ruigang Yang,et al.  Learning Warped Guidance for Blind Face Restoration , 2018, ECCV.

[28]  Stefanos Zafeiriou,et al.  ArcFace: Additive Angular Margin Loss for Deep Face Recognition , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Jiri Matas,et al.  DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[30]  Alberto Del Bimbo,et al.  Deep Generative Adversarial Compression Artifact Removal , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[31]  Yu Qiao,et al.  Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks , 2016, IEEE Signal Processing Letters.

[32]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[33]  B. K. Joshi,et al.  A Review Paper: Noise Models in Digital Image Processing , 2015, ArXiv.

[34]  Jean Ponce,et al.  Learning a convolutional neural network for non-uniform motion blur removal , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Aaron C. Courville,et al.  Generative adversarial networks , 2014, Commun. ACM.

[36]  Q. M. Jonathan Wu,et al.  Low-resolution face recognition: a review , 2013, The Visual Computer.

[37]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[38]  P. Corcoran,et al.  iFace 1.1: A Proof-of-Concept of a Facial Authentication Based Digital ID for Smart Cities , 2022, IEEE Access.

[39]  Menglei Zhang,et al.  Supervised Pixel-Wise GAN for Face Super-Resolution , 2021, IEEE Transactions on Multimedia.

[40]  Shamik Tiwari,et al.  A Blur Classification Approach Using Deep Convolution Neural Network , 2020, Int. J. Inf. Syst. Model. Des..

[41]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.