A review on face recognition systems: recent approaches and challenges

Face recognition is an efficient technique and one of the most preferred biometric modalities for the identification and verification of individuals as compared to voice, fingerprint, iris, retina eye scan, gait, ear and hand geometry. This has over the years necessitated researchers in both the academia and industry to come up with several face recognition techniques making it one of the most studied research area in computer vision. A major reason why it remains a fast-growing research lies in its application in unconstrained environments, where most existing techniques do not perform optimally. Such conditions include pose, illumination, ageing, occlusion, expression, plastic surgery and low resolution. In this paper, a critical review on the different issues of face recognition systems are presented, and different approaches to solving these issues are analyzed by presenting existing techniques that have been proposed in the literature. Furthermore, the major and challenging face datasets that consist of the different facial constraints which depict real-life scenarios are also discussed stating the shortcomings associated with them. Also, recognition performance on the different datasets by researchers are also reported. The paper is concluded, and directions for future works are highlighted.

[1]  Edilson de Aguiar,et al.  Facial expression recognition with Convolutional Neural Networks: Coping with few data and the training sample order , 2017, Pattern Recognit..

[2]  Bin Fang,et al.  Extracting sparse error of robust PCA for face recognition in the presence of varying illumination and occlusion , 2014, Pattern Recognit..

[3]  Shengcai Liao,et al.  A benchmark study of large-scale unconstrained face recognition , 2014, IEEE International Joint Conference on Biometrics.

[4]  Marc'Aurelio Ranzato,et al.  Building high-level features using large scale unsupervised learning , 2011, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[5]  Sheng-Luen Chung,et al.  Robust cross-pose face recognition using landmark oriented depth warping , 2018, J. Vis. Commun. Image Represent..

[6]  Robert H. Riffenburgh,et al.  Linear Discriminant Analysis , 1960 .

[7]  Yu-Feng Yu,et al.  Discriminative multi-layer illumination-robust feature extraction for face recognition , 2017, Pattern Recognit..

[8]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[9]  Dakshina Ranjan Kisku,et al.  Face Identification via Strategic Combination of Local Features , 2020 .

[10]  Yuli Fu,et al.  Efficient locality-constrained occlusion coding for face recognition , 2017, Neurocomputing.

[11]  Gerhard P. Hancke,et al.  Unimodal and Multimodal Biometric Sensing Systems: A Review , 2016, IEEE Access.

[12]  Xin Jin,et al.  Face alignment in-the-wild: A Survey , 2016, Comput. Vis. Image Underst..

[13]  Chris Rowen,et al.  Using Convolutional Neural Networks for Image Recognition By , 2015 .

[14]  Oliveira-SantosThiago,et al.  Facial expression recognition with Convolutional Neural Networks , 2017 .

[15]  Ling Shao,et al.  Robust Face Recognition With Kernelized Locality-Sensitive Group Sparsity Representation , 2017, IEEE Transactions on Image Processing.

[16]  Dima Damen,et al.  Recognizing linked events: Searching the space of feasible explanations , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Michele Nappi,et al.  Deceiving faces: When plastic surgery challenges face recognition , 2016, Image Vis. Comput..

[18]  Patrick J. Flynn,et al.  A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition , 2006, Comput. Vis. Image Underst..

[19]  Li De-ren Research on Particle Swarm Optimization in Remote Sensing Image Enhancement , 2010 .

[20]  Gerhard P. Hancke,et al.  A new evaluation function for face image enhancement in unconstrained environments using metaheuristic algorithms , 2019, EURASIP J. Image Video Process..

[21]  Mazdak Zamani,et al.  Face Recognition via Taxonomy of Illumination Normalization , 2017 .

[22]  Jian Yang,et al.  Learning robust and discriminative low-rank representations for face recognition with occlusion , 2017, Pattern Recognit..

[23]  Haider A. Alwzwazy,et al.  Robust Convolutional Neural Networks for Image Recognition , 2015 .

[24]  Jin Li,et al.  Privacy-preserving Naive Bayes classifiers secure against the substitution-then-comparison attack , 2018, Inf. Sci..

[25]  Zhenxue Chen,et al.  Illumination and pose variable face recognition via adaptively weighted ULBP_MHOG and WSRC , 2017, Signal Process. Image Commun..

[26]  Rolf P. Würtz,et al.  Elastic Bunch Graph Matching , 2014, Scholarpedia.

[27]  Weisheng Li,et al.  Pose-robust face recognition with Huffman-LBP enhanced by Divide-and-Rule strategy , 2018, Pattern Recognit..

[28]  Ming Zhu,et al.  A Novel Two-stage Learning Pipeline for Deep Neural Networks , 2017, Neural Processing Letters.

[29]  Andrea F. Abate,et al.  2D and 3D face recognition: A survey , 2007, Pattern Recognit. Lett..

[30]  Jing-Wein Wang,et al.  Illumination compensation for face recognition using adaptive singular value decomposition in the wavelet domain , 2018, Inf. Sci..

[31]  Arman Savran,et al.  Computer Vision and Image Understanding , 2022, SSRN Electronic Journal.

[32]  Aamir Saeed Malik,et al.  Proposed face recognition system after plastic surgery , 2016, IET Comput. Vis..

[33]  Jun Guo,et al.  Face Recognition via Collaborative Representation: Its Discriminant Nature and Superposed Representation , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Weihong Deng,et al.  Supplementary Material for Unsupervised Face Normalization with Extreme Pose and Expression in the Wild , 2019 .

[35]  João M. F. Rodrigues,et al.  Expression-invariant face recognition using a biological disparity energy model , 2018, Neurocomputing.

[36]  Allen Y. Yang,et al.  Sparse Illumination Learning and Transfer for Single-Sample Face Recognition with Image Corruption and Misalignment , 2014, International Journal of Computer Vision.

[37]  Stefanos Zafeiriou,et al.  A survey on face detection in the wild: Past, present and future , 2015, Comput. Vis. Image Underst..

[38]  Kin-Man Lam,et al.  Age-invariant face recognition based on identity inference from appearance age , 2018, Pattern Recognit..

[39]  Anastasios Tefas,et al.  Variants of dynamic link architecture based on mathematical morphology for frontal face authentication , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[40]  Peng An,et al.  Application of robust face recognition in video surveillance systems , 2018 .

[41]  Ming Li,et al.  2D-LDA: A statistical linear discriminant analysis for image matrix , 2005, Pattern Recognit. Lett..

[42]  Mislav Grgic,et al.  Independent comparative study of PCA, ICA, and LDA on the FERET data set , 2005, Int. J. Imaging Syst. Technol..

[43]  Wen Hao,et al.  Expression-insensitive 3D face recognition by the fusion of multiple subject-specific curves , 2018, Neurocomputing.

[44]  张德馨,et al.  Application of robust face recognition in video surveillance systems , 2018 .

[45]  Shuhuan Zhao,et al.  Pixel-level occlusion detection based on sparse representation for face recognition , 2018, Optik.

[46]  V. Kshirsagar,et al.  Face recognition using Eigenfaces , 2011, 2011 3rd International Conference on Computer Research and Development.

[47]  R. S. Jadon,et al.  Retraction Note to: Real-Life Facial Expression Recognition Systems: A Review , 2019 .

[48]  Xueming Qian,et al.  Robust Framework of Single-Frame Face Superresolution Across Head Pose, Facial Expression, and Illumination Variations , 2015, IEEE Transactions on Human-Machine Systems.

[49]  Mebarka Belahcene,et al.  3D face recognition in presence of expressions by fusion regions of interest , 2014, 2014 22nd Signal Processing and Communications Applications Conference (SIU).

[50]  Roberto Brunelli,et al.  Face Recognition: Features Versus Templates , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[51]  Ghazali Sulong,et al.  Face recognition via edge-based Gabor feature representation for plastic surgery-altered images , 2014, EURASIP J. Adv. Signal Process..

[52]  Josef Kittler,et al.  Face Detection, Bounding Box Aggregation and Pose Estimation for Robust Facial Landmark Localisation in the Wild , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[53]  Kai Zhao,et al.  RegularFace: Deep Face Recognition via Exclusive Regularization , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[54]  Xavier Maldague,et al.  Infrared face recognition: A comprehensive review of methodologies and databases , 2014, Pattern Recognit..

[55]  Dacheng Tao,et al.  Pose-invariant face recognition with homography-based normalization , 2017, Pattern Recognit..

[56]  Jiliu Zhou,et al.  Geometric feature based facial expression recognition using multiclass support vector machines , 2009, 2009 IEEE International Conference on Granular Computing.

[57]  Ling Shao,et al.  Face recognition with a small occluded training set using spatial and statistical pooling , 2018, Inf. Sci..

[58]  ZhangZhengyou,et al.  A survey on face detection in the wild , 2015 .

[59]  Jing-Yu Yang,et al.  Illumination invariant extraction for face recognition using neighboring wavelet coefficients , 2012, Pattern Recognit..

[60]  Marios Savvides,et al.  Face Recognition Across Pose Using View Based Active Appearance Models (VBAAMs) on CMU Multi-PIE Dataset , 2008, ICVS.

[61]  Ioannis Pitas,et al.  Face Authentication Using Morphological Dynamic Link Architecture , 1997, AVBPA.

[62]  Sanjay N. Talbar,et al.  Recognition of plastic surgery faces and the surgery types: An approach with entropy based scale invariant features , 2019, J. King Saud Univ. Comput. Inf. Sci..

[63]  Hossein Mobahi,et al.  Face recognition with contiguous occlusion using markov random fields , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[64]  Chokri Ben Amar,et al.  A Survey of 2D Face Recognition Techniques , 2016, Comput..

[65]  Gerhard P. Hancke,et al.  Evaluating the effect of occlusion in face recognition systems , 2017, 2017 IEEE AFRICON.

[66]  WangXiaogang,et al.  A survey on heterogeneous face recognition , 2016 .

[67]  Xuelong Li,et al.  Aging Face Recognition: A Hierarchical Learning Model Based on Local Patterns Selection , 2016, IEEE Transactions on Image Processing.

[68]  Madasu Hanmandlu,et al.  Face recognition using Elastic bunch graph matching , 2013, 2013 IEEE Applied Imagery Pattern Recognition Workshop (AIPR).

[69]  Yang Gao,et al.  Local histogram specification for face recognition under varying lighting conditions , 2014, Image Vis. Comput..

[70]  Ying Tai,et al.  Nuclear Norm Based Matrix Regression with Applications to Face Recognition with Occlusion and Illumination Changes , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[71]  Jianping Gou,et al.  An antinoise sparse representation method for robust face recognition via joint l1 and l2 regularization , 2017, Expert Syst. Appl..

[72]  W. R. Sam Emmanuel,et al.  Face expression recognition using LDN and Dominant Gradient Local Ternary Pattern descriptors , 2018, J. King Saud Univ. Comput. Inf. Sci..

[73]  Hung-Hsu Tsai,et al.  Facial expression recognition using a combination of multiple facial features and support vector machine , 2018, Soft Comput..

[74]  Sergio Escalera,et al.  Convolutional Neural Network Super Resolution for Face Recognition in Surveillance Monitoring , 2016, AMDO.

[75]  Timothy Hospedales,et al.  A survey on heterogeneous face recognition: Sketch, infra-red, 3D and low-resolution , 2014, Image Vis. Comput..

[76]  Rashmi Gupta,et al.  Human identification after plastic surgery using region based score level fusion of local facial features , 2019, J. Inf. Secur. Appl..

[77]  이창기,et al.  Convolutional Neural Network를 이용한 한국어 영화평 감성 분석 , 2016 .

[78]  Wei-Ping Zhu,et al.  Face recognition via fast dense correspondence , 2018, Multimedia Tools and Applications.

[79]  Mao Ye,et al.  Age invariant face recognition and retrieval by coupled auto-encoder networks , 2017, Neurocomputing.

[80]  Chin-Teng Lin,et al.  Deep Sparse Representation Classifier for facial recognition and detection system , 2019, Pattern Recognit. Lett..

[81]  Yong Shi,et al.  Robust twin support vector machine for pattern classification , 2013, Pattern Recognit..

[82]  Ke Xiao,et al.  A Multi-Scale Parallel Convolutional Neural Network Based Intelligent Human Identification Using Face Information , 2018, J. Inf. Process. Syst..

[83]  Shanmukhappa A. Angadi,et al.  A robust face recognition approach through symbolic modeling of Polar FFT features , 2017, Pattern Recognit..

[84]  Amjad Rehman,et al.  Neural networks for document image preprocessing: state of the art , 2014, Artificial Intelligence Review.

[85]  Di Huang,et al.  Local Binary Patterns and Its Application to Facial Image Analysis: A Survey , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[86]  Kosin Chamnongthai,et al.  A pose and expression face recognition method using transformation based on single face neutral reference , 2017, 2017 Global Wireless Summit (GWS).

[87]  Patrick J. Flynn,et al.  Template aging in 3D and 2D face recognition , 2016, 2016 IEEE 8th International Conference on Biometrics Theory, Applications and Systems (BTAS).

[88]  Huaming Wu,et al.  Learning deep discriminative face features by customized weighted constraint , 2019, Neurocomputing.

[89]  Zhengtao Yu,et al.  Locality Preserving Collaborative Representation for Face Recognition , 2017, Neural Processing Letters.

[90]  Yongjie Chu,et al.  Low-resolution face recognition with single sample per person , 2017, Signal Process..

[91]  Angelo Cangelosi,et al.  Head pose estimation in the wild using Convolutional Neural Networks and adaptive gradient methods , 2017, Pattern Recognit..

[92]  Nello Cristianini,et al.  Gender Classification by Deep Learning on Millions of Weakly Labelled Images , 2016, 2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW).

[93]  George D. C. Cavalcanti,et al.  A robust feature extraction algorithm based on class-Modular Image Principal Component Analysis for face verification , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[94]  Gérard G. Medioni,et al.  Pose-Aware Face Recognition in the Wild , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[95]  Yang Lu,et al.  Face Occlusion Detection Using Cascaded Convolutional Neural Network , 2016, CCBR.

[96]  Gouhei Tanaka,et al.  A Hybrid Pooling Method for Convolutional Neural Networks , 2016, ICONIP.

[97]  Yiming Zhang,et al.  Coupled marginal discriminant mappings for low-resolution face recognition , 2015 .

[98]  Yuxiao Hu,et al.  MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition , 2016, ECCV.

[99]  Marian Stewart Bartlett,et al.  Face recognition by independent component analysis , 2002, IEEE Trans. Neural Networks.

[100]  Andrea Cavallaro,et al.  Automatic Analysis of Facial Affect: A Survey of Registration, Representation, and Recognition , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[101]  Tao Liu,et al.  Multi-step linear representation-based classification for face recognition , 2016, IET Comput. Vis..

[102]  Gerhard P. Hancke,et al.  Improving Face Recognition Systems Using a New Image Enhancement Technique, Hybrid Features and the Convolutional Neural Network , 2018, IEEE Access.

[103]  R. S. Jadon,et al.  RETRACTED CHAPTER: Real-Life Facial Expression Recognition Systems: A Review , 2018 .

[104]  Haifeng Hu,et al.  ICA-based neighborhood preserving analysis for face recognition , 2008, Comput. Vis. Image Underst..

[105]  Nuno Vasconcelos,et al.  PIEs: Pose Invariant Embeddings , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[106]  Hassen Drira,et al.  3D Face Recognition under Expressions, Occlusions, and Pose Variations , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[107]  M. Arif Wani,et al.  Elastic Bunch Graph Matching Based Face Recognition Under Varying Lighting, Pose, and Expression Conditions , 2016 .

[108]  Philippe De Wilde,et al.  Recognizing faces prone to occlusions and common variations using optimal face subgraphs , 2016, Appl. Math. Comput..

[109]  Yong Liu,et al.  Multi-modal force/vision sensor fusion in 6-DOF pose tracking , 2009, 2009 International Conference on Advanced Robotics.

[110]  Shasha Wang,et al.  Deep feature weighting for naive Bayes and its application to text classification , 2016, Eng. Appl. Artif. Intell..

[111]  Dongqing Zhang,et al.  Neural Aggregation Network for Video Face Recognition , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[112]  Joachim M. Buhmann,et al.  Distortion Invariant Object Recognition in the Dynamic Link Architecture , 1993, IEEE Trans. Computers.

[113]  Richa Singh,et al.  On Matching Faces with Alterations due to Plastic Surgery and Disguise , 2018, 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS).

[114]  Ioannis A. Kakadiaris,et al.  3D-2D face recognition with pose and illumination normalization , 2017, Comput. Vis. Image Underst..