Multiscale parallel deep CNN (mpdCNN) architecture for the real low-resolution face recognition for surveillance

Abstract Images from the surveillance networks are being extensively used for the purpose of monitoring and criminal investigations. However, it is often difficult to recognize faces using the data from the surveillance networks because the resolution of the face in the images captured is too low. Further, the low-resolution images have varying magnitude of facial feature content because of wide variations in illumination, pose, resolution and the distance from which the image is captured. Also, a single face recognition solution is not able to recognize faces efficiently in both high and low-resolution images. Wide variations in facial feature content in high and low-resolution images causes difficulty in classification of the features by a single model for the purpose of face recognition. We present a Deep-CNN based architecture called mpdCNN to solve the problem of face recognition in low as well as high-resolution images with high accuracy and robustness. Our proposed architecture mpdCNN gives 88.6% accuracy on the SCface database which is an impressive improvement over the state-of-the-art algorithms. We also achieved an accuracy of above 99% on normal to high-resolution databases for face recognition.

[1]  Shiv Ram Dubey,et al.  Hard-Mining Loss based Convolutional Neural Network for Face Recognition , 2019, ArXiv.

[2]  Pong C. Yuen,et al.  Very low resolution face recognition problem , 2010, 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[3]  Nasser M. Nasrabadi,et al.  Coupled generative adversarial network for heterogeneous face recognition , 2020, Image Vis. Comput..

[4]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Luuk J. Spreeuwers,et al.  Designing a Low-Resolution Face Recognition System for Long-Range Surveillance , 2016, 2016 International Conference of the Biometrics Special Interest Group (BIOSIG).

[6]  David A. Forsyth,et al.  Shape, Contour and Grouping in Computer Vision , 1999, Lecture Notes in Computer Science.

[7]  Dong Yue,et al.  Low-Rank Representation and Locality-Constrained Regression for Robust Low-Resolution Face Recognition , 2018 .

[8]  Yücel Altunbasak,et al.  Eigenface-domain super-resolution for face recognition , 2003, IEEE Trans. Image Process..

[9]  Fabrizio Falchi,et al.  Cross-Resolution Learning for Face Recognition , 2020, Image Vis. Comput..

[10]  Conrad Sanderson,et al.  The VidTIMIT Database , 2002 .

[11]  Tao Xiang,et al.  Multi-Scale Learning for Low-Resolution Person Re-Identification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[12]  Mislav Grgic,et al.  SCface – surveillance cameras face database , 2011, Multimedia Tools and Applications.

[13]  Patrick J. Flynn,et al.  Eye Perturbation Approach for Robust Recognition of Inaccurately Aligned Faces , 2005, AVBPA.

[14]  Shiming Ge,et al.  Low-Resolution Face Recognition in the Wild via Selective Knowledge Distillation , 2018, IEEE Transactions on Image Processing.

[15]  Wen-Ming Cao,et al.  Face recognition based on manifold learning and Rényi entropy , 2010 .

[16]  Wafaa Shihab Ahmed,et al.  The Impact of Filter Size and Number of Filters on Classification Accuracy in CNN , 2020, 2020 International Conference on Computer Science and Software Engineering (CSASE).

[17]  Patrick J. Flynn,et al.  On Low-Resolution Face Recognition in the Wild: Comparisons and New Techniques , 2018, IEEE Transactions on Information Forensics and Security.

[18]  Patrick J. Flynn,et al.  Learning face similarity for re-identification from real surveillance video: A deep metric solution , 2017, 2017 IEEE International Joint Conference on Biometrics (IJCB).

[19]  Masashi Nishiyama,et al.  Low-resolution person recognition using image downsampling , 2017, 2017 Fifteenth IAPR International Conference on Machine Vision Applications (MVA).

[20]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[21]  Sivaram Prasad Mudunuri,et al.  GenLR-Net: Deep Framework for Very Low Resolution Face and Object Recognition with Generalization to Unseen Categories , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[22]  Chi-Ho Chan,et al.  Resolution Invariant Face Recognition Using a Distillation Approach , 2020, IEEE Transactions on Biometrics, Behavior, and Identity Science.

[23]  Shiv Ram Dubey,et al.  A Performance Evaluation of Loss Functions for Deep Face Recognition , 2020 .

[24]  Paul Honeine,et al.  A hierarchical classification method using belief functions , 2018, Signal Process..

[25]  Shaogang Gong,et al.  Low-Resolution Face Recognition , 2018, ACCV.

[26]  Takeo Kanade,et al.  Hallucinating faces , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[27]  Nayaneesh Kumar Mishra,et al.  Face Recognition using 3D CNNs , 2021, ArXiv.

[28]  Zhong Jin,et al.  Down-Sampling Face Images and Low-Resolution Face Recognition , 2008, 2008 3rd International Conference on Innovative Computing Information and Control.

[29]  Qingmin Liao,et al.  Discriminative Multidimensional Scaling for Low-Resolution Face Recognition , 2018, IEEE Signal Processing Letters.

[30]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Rama Chellappa,et al.  Synthesis-based recognition of low resolution faces , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[32]  Zied Lachiri,et al.  Video-Based Face Recognition Using HMM and Details Horizontal Orientation by 2D-DWT: Application to VIDTIMIT Database , 2016 .

[33]  Xudong Jiang,et al.  Deep Coupled ResNet for Low-Resolution Face Recognition , 2018, IEEE Signal Processing Letters.

[34]  Thomas S. Huang,et al.  Studying Very Low Resolution Recognition Using Deep Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Hua Huang,et al.  Super-Resolution Method for Face Recognition Using Nonlinear Mappings on Coherent Features , 2011, IEEE Transactions on Neural Networks.

[36]  Patrick J. Flynn,et al.  Toward facial re-identification: Experiments with data from an operational surveillance camera plant , 2016, 2016 IEEE 8th International Conference on Biometrics Theory, Applications and Systems (BTAS).

[37]  Zheng Wang,et al.  SRLSP: A Face Image Super-Resolution Algorithm Using Smooth Regression With Local Structure Prior , 2017, IEEE Transactions on Multimedia.

[38]  Shengcai Liao,et al.  Coupled Discriminant Analysis for Heterogeneous Face Recognition , 2012, IEEE Transactions on Information Forensics and Security.

[39]  Hong Yan,et al.  Coupled Kernel Embedding for Low-Resolution Face Image Recognition , 2012, IEEE Transactions on Image Processing.

[40]  Wen Gao,et al.  The CAS-PEAL Large-Scale Chinese Face Database and Baseline Evaluations , 2008, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[41]  Shiguang Shan,et al.  Low-Resolution Face Recognition via Coupled Locality Preserving Mappings , 2010, IEEE Signal Processing Letters.

[42]  Ioannis A. Kakadiaris,et al.  Semi-coupled basis and distance metric learning for cross-domain matching: Application to low-resolution face recognition , 2014, IEEE International Joint Conference on Biometrics.

[43]  Victor Hugo C. de Albuquerque,et al.  A high-efficiency energy and storage approach for IoT applications of facial recognition , 2020, Image Vis. Comput..

[44]  Luuk J. Spreeuwers,et al.  Low-resolution face alignment and recognition using mixed-resolution classifiers , 2017, IET Biom..