Low Resolution Face Recognition Using a Two-Branch Deep Convolutional Neural Network Architecture

We propose a novel couple mappings method for low resolution face recognition using deep convolutional neural networks (DCNNs). The proposed architecture consists of two branches of DCNNs to map the high and low resolution face images into a common space with nonlinear transformations. The branch corresponding to transformation of high resolution images consists of 14 layers and the other branch which maps the low resolution face images to the common space includes a 5-layer super-resolution network connected to a 14-layer network. The distance between the features of corresponding high and low resolution images are backpropagated to train the networks. Our proposed method is evaluated on FERET data set and compared with state-of-the-art competing methods. Our extensive experimental results show that the proposed method significantly improves the recognition performance especially for very low resolution probe face images (11.4% improvement in recognition accuracy). Furthermore, it can reconstruct a high resolution image from its corresponding low resolution probe image which is comparable with state-of-the-art super-resolution methods in terms of visual quality.

[1]  Xiaogang Wang,et al.  Deep Learning Face Representation from Predicting 10,000 Classes , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Patrick J. Flynn,et al.  On Low-Resolution Face Recognition in the Wild: Comparisons and New Techniques , 2018, IEEE Transactions on Information Forensics and Security.

[3]  Kin-Man Lam,et al.  Simultaneous Hallucination and Recognition of Low-Resolution Faces Based on Singular Value Decomposition , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[4]  Shiguang Shan,et al.  Low-Resolution Face Recognition via Coupled Locality Preserving Mappings , 2010, IEEE Signal Processing Letters.

[5]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.

[7]  Hyeonjoon Moon,et al.  The FERET Evaluation Methodology for Face-Recognition Algorithms , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Shree K. Nayar,et al.  Attribute and simile classifiers for face verification , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[9]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[10]  Harry Shum,et al.  Face Hallucination: Theory and Practice , 2007, International Journal of Computer Vision.

[11]  Rama Chellappa,et al.  Super-Resolution of Face Images Using Kernel PCA-Based Prior , 2007, IEEE Transactions on Multimedia.

[12]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[13]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[14]  Takeo Kanade,et al.  Hallucinating faces , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[15]  Yiming Zhang,et al.  Coupled marginal discriminant mappings for low-resolution face recognition , 2015 .

[16]  Hongjun Jia,et al.  Support Vector Machines in face recognition with occlusions , 2009, CVPR.

[17]  Sivaram Prasad Mudunuri,et al.  Dictionary Alignment for Low-Resolution and Heterogeneous Face Recognition , 2017, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[18]  Domingo Mery,et al.  Face recognition in low-quality images using adaptive sparse representations , 2019, Image Vis. Comput..

[19]  Stefanos Zafeiriou,et al.  300 Faces in-the-Wild Challenge: The First Facial Landmark Localization Challenge , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[20]  Xudong Jiang,et al.  Deep Coupled ResNet for Low-Resolution Face Recognition , 2018, IEEE Signal Processing Letters.

[21]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Hazim Kemal Ekenel,et al.  Exploring Factors for Improving Low Resolution Face Recognition , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[23]  Andrew Zisserman,et al.  Deep Face Recognition , 2015, BMVC.

[24]  Qingmin Liao,et al.  Discriminative Multidimensional Scaling for Low-Resolution Face Recognition , 2018, IEEE Signal Processing Letters.

[25]  Deva Ramanan,et al.  Face detection, pose estimation, and landmark localization in the wild , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Stan Z. Li,et al.  Low-resolution face recognition via Simultaneous Discriminant Analysis , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[27]  Dahua Lin,et al.  Hallucinating faces: TensorPatch super-resolution and coupled residue compensation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[28]  Dacheng Tao,et al.  Robust Face Recognition via Multimodal Deep Face Representation , 2015, IEEE Transactions on Multimedia.

[29]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[30]  Aristodemos Pnevmatikakis,et al.  Far-Field, Multi-Camera, Video-to-Video Face Recognition , 2007 .

[31]  Xin Yu,et al.  Super-Resolving Very Low-Resolution Face Images with Supplementary Attributes , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[32]  Bruce A. Draper,et al.  Overview of the Multiple Biometrics Grand Challenge , 2009, ICB.

[33]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[34]  Thomas S. Huang,et al.  Interactive Facial Feature Localization , 2012, ECCV.

[35]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[36]  Sivaram Prasad Mudunuri,et al.  Low Resolution Face Recognition Across Variations in Pose and Illumination , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Pablo H. Hennings-Yeomans,et al.  Simultaneous super-resolution and feature extraction for recognition of low-resolution faces , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  D. B. Graham,et al.  Characterising Virtual Eigensignatures for General Purpose Face Recognition , 1998 .

[41]  Hong Yan,et al.  Coupled Kernel Embedding for Low-Resolution Face Image Recognition , 2012, IEEE Transactions on Image Processing.

[42]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[43]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[44]  Sepp Hochreiter,et al.  Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) , 2015, ICLR.

[45]  Hua Huang,et al.  Super-Resolution Method for Face Recognition Using Nonlinear Mappings on Coherent Features , 2011, IEEE Transactions on Neural Networks.

[46]  Chun Qi,et al.  From Local Geometry to Global Structure: Learning Latent Subspace for Low-resolution Face Image Recognition , 2015, IEEE Signal Processing Letters.

[47]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[48]  Thierry Pun,et al.  A Stochastic Approach to Content Adaptive Digital Image Watermarking , 1999, Information Hiding.

[49]  尚弘 島影 National Institute of Standards and Technologyにおける超伝導研究及び生活 , 2001 .

[50]  Patrick J. Flynn,et al.  Multidimensional Scaling for Matching Low-Resolution Face Images , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[52]  Xiaogang Wang,et al.  Hybrid Deep Learning for Face Verification , 2013, ICCV.

[53]  Horst Bischof,et al.  Large scale metric learning from equivalence constraints , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[54]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[55]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[56]  William T. Freeman,et al.  Learning Low-Level Vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[57]  Luc Van Gool,et al.  The Pascal Visual Object Classes Challenge: A Retrospective , 2014, International Journal of Computer Vision.

[58]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[59]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[60]  Pong C. Yuen,et al.  Very low resolution face recognition problem , 2010, 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[61]  Alex Pentland,et al.  Face recognition using eigenfaces , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[62]  Q. M. Jonathan Wu,et al.  Low-resolution face recognition: a review , 2013, The Visual Computer.

[63]  Aleix M. Martínez,et al.  Recognizing Imprecisely Localized, Partially Occluded, and Expression Variant Faces from a Single Sample per Class , 2002, IEEE Trans. Pattern Anal. Mach. Intell..