Does face restoration improve face verification?

Methods for face verification works reasonably well on face images with standardized (frontal) face positions and good spatial resolution. However such methods have significant challenges on poor resolution images, poor lighting conditions and not standard (frontal) face positions. In this paper, we survey the capability of existing face restoration and verification methods, with the aim of understanding how useful face restoration methods are for face verification. We propose a qualitative and quantitative comparison benchmark, and apply it on eight methods for face restoration and six methods for face verification, on several real-world low-quality images from a surveillance context, and outline observed advantages and limitations. Experiments shows that each restoration method can affect each face verification method differently, with fewer than the half of face restoration methods helping face verification. Interestingly, some face restoration methods with less good qualitative evaluation helped face verification the most. Experiments also show that face verification works less good if the resolution decreases.

[1]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[2]  Renjie Liao,et al.  Detail-Revealing Deep Video Super-Resolution , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[3]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[4]  Zhengyou Zhang,et al.  A Survey of Recent Advances in Face Detection , 2010 .

[5]  K. Egiazarian,et al.  Blind image deconvolution , 2007 .

[6]  R. S,et al.  ID Photo Verification by Face Recognition , 2020, 2020 6th International Conference on Advanced Computing and Communication Systems (ICACCS).

[7]  Yu Qiao,et al.  Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks , 2016, IEEE Signal Processing Letters.

[8]  Xiaogang Wang,et al.  Deep Learning Face Representation from Predicting 10,000 Classes , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Djemel Ziou,et al.  Image Quality Metrics: PSNR vs. SSIM , 2010, 2010 20th International Conference on Pattern Recognition.

[10]  Li WangDong-Chen He,et al.  Texture classification using texture spectrum , 1990, Pattern Recognit..

[11]  R. Keys.  Cubic convolution interpolation for digital image processing , 1981 .

[12]  J KriegmanDavid,et al.  Eigenfaces vs. Fisherfaces , 1997 .

[13]  Narendra Ahuja,et al.  Single image super-resolution from transformed self-exemplars , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Jian Cheng,et al.  NormFace: L2 Hypersphere Embedding for Face Verification , 2017, ACM Multimedia.

[16]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Yu Qiao,et al.  A Discriminative Feature Learning Approach for Deep Face Recognition , 2016, ECCV.

[18]  Wangmeng Zuo,et al.  Learning a Single Convolutional Super-Resolution Network for Multiple Degradations , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[19]  Shengcai Liao,et al.  A benchmark study of large-scale unconstrained face recognition , 2014, IEEE International Joint Conference on Biometrics.

[20]  Katsushi Ikeuchi,et al.  Simultaneous deblur and super-resolution technique for video sequence captured by hand-held video camera , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[21]  Joseph Redmon,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[22]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[23]  Matthew A. Brown,et al.  Frame-Recurrent Video Super-Resolution , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[24]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Jian Yang,et al.  FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[26]  Bhiksha Raj,et al.  SphereFace: Deep Hypersphere Embedding for Face Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Tieniu Tan,et al.  A Light CNN for Deep Face Representation With Noisy Labels , 2015, IEEE Transactions on Information Forensics and Security.

[28]  Kyoung Mu Lee,et al.  Deeply-Recursive Convolutional Network for Image Super-Resolution , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Patrick Ogao,et al.  Classifying desirable features of software visualization tools for corrective maintenance , 2008, SOFTVIS.

[30]  Narendra Ahuja,et al.  Detecting Faces in Images: A Survey , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  S. Carey,et al.  From piecemeal to configurational representation of faces. , 1977, Science.

[32]  Jiri Matas,et al.  DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[33]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Lu Zhang,et al.  Multi‐view frontal face image generation: A survey , 2020, Concurr. Comput. Pract. Exp..

[35]  Alexandru Telea,et al.  Joint Brightness and Tone Stabilization of Capsule Endoscopy Videos , 2018, VISIGRAPP.

[36]  Bo Tang,et al.  BULDP: Biomimetic Uncorrelated Locality Discriminant Projection for Feature Extraction in Face Recognition , 2018, IEEE Transactions on Image Processing.

[37]  Marek R. Ogiela,et al.  Multimedia tools and applications , 2005, Multimedia Tools and Applications.

[38]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[39]  Gang Hua,et al.  Labeled Faces in the Wild: A Survey , 2016 .

[40]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Ziyou Xiong,et al.  A two-step face hallucination approach for video surveillance applications , 2013, Multimedia Tools and Applications.