Deep Learning meets Liveness Detection: Recent Advancements and Challenges

Facial biometrics has been recently received tremendous attention as a convenient replacement for traditional authentication systems. Consequently, detecting malicious attempts has found great significance, leading to extensive studies in face anti-spoofing (FAS),i.e., face presentation attack detection. Deep feature learning and techniques, as opposed to hand-crafted features, have promised dramatic increase in the FAS systems’ accuracy, tackling the key challenges of materializing realworld application of such systems. Hence, a new research area dealing with development of more generalized as well as accurate models is increasingly attracting the attention of the research community and industry. In this paper, we present a comprehensive survey on the literature related to deep-feature-based FAS methods since 2017. To shed light on this topic, a semantic taxonomy based on various features and learning methodologies is represented. Further, we cover predominant public datasets for FAS in a chronological order, their evolutional progress, and the evaluation criteria (both intra-dataset and inter-dataset). Finally, we discuss the open research challenges and future directions.

[1]  Xiaoyue Jiang,et al.  Face Anti-spoofing via Deep Local Binary Pattern , 2017, Deep Learning in Object Detection and Recognition.

[2]  Rongrong Ji,et al.  Aurora Guard: Real-Time Face Anti-Spoofing via Light Reflection , 2019, ArXiv.

[3]  William T. Freeman,et al.  Orientation Histograms for Hand Gesture Recognition , 1995 .

[4]  Feiyue Huang,et al.  Face Anti-Spoofing Via Disentangled Representation Learning , 2020, ECCV.

[5]  Kavitha S.N,et al.  Face Anti-spoofing Technique Using CNN and SVM , 2019, 2019 International Conference on Intelligent Computing and Control Systems (ICCS).

[6]  Pong C. Yuen,et al.  Multi-Adversarial Discriminative Deep Domain Generalization for Face Presentation Attack Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Vishal M. Patel,et al.  Anomaly Detection-Based Unknown Face Presentation Attack Detection , 2020, 2020 IEEE International Joint Conference on Biometrics (IJCB).

[9]  Anjith George,et al.  On the Effectiveness of Vision Transformers for Zero-shot Face Anti-Spoofing , 2020, 2021 IEEE International Joint Conference on Biometrics (IJCB).

[10]  Shiliang Pu,et al.  Self-Domain Adaptation for Face Anti-Spoofing , 2021, AAAI.

[11]  Omkar M. Parkhi,et al.  VGGFace2: A Dataset for Recognising Faces across Pose and Age , 2017, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[12]  Anjith George,et al.  Learning One Class Representations for Face Presentation Attack Detection Using Multi-Channel Convolutional Neural Networks , 2020, IEEE Transactions on Information Forensics and Security.

[13]  Zezheng Wang,et al.  One-class adaptation face anti-spoofing with loss function search , 2020, Neurocomputing.

[14]  Zhaoqiang Xia,et al.  Revisiting Pixel-Wise Supervision for Face Anti-Spoofing , 2020, IEEE Transactions on Biometrics, Behavior, and Identity Science.

[15]  Xiaoming Liu,et al.  Deep Tree Learning for Zero-Shot Face Anti-Spoofing , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Yuchun Fang,et al.  Ultra-deep Neural Network for Face Anti-spoofing , 2017, ICONIP.

[17]  Javier Hernandez-Ortega,et al.  Time Analysis of Pulse-Based Face Anti-Spoofing in Visible and NIR , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[18]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[19]  Yu Song,et al.  Face Spoofing Detection Based on Local Ternary Label Supervision in Fully Convolutional Networks , 2020, IEEE Transactions on Information Forensics and Security.

[20]  Sébastien Marcel,et al.  On the effectiveness of local binary patterns in face anti-spoofing , 2012, 2012 BIOSIG - Proceedings of the International Conference of Biometrics Special Interest Group (BIOSIG).

[21]  Kwok-Yan Lam,et al.  Unseen Face Presentation Attack Detection with Hypersphere Loss , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[22]  Sébastien Marcel,et al.  Counter-measures to photo attacks in face recognition: A public database and a baseline , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[23]  Weihong Deng,et al.  Deep Local Descriptors with Domain Adaptation , 2018, PRCV.

[24]  Xiaoming Liu,et al.  Face De-Spoofing: Anti-Spoofing via Noise Modeling , 2018, ECCV.

[25]  Yue Zhao,et al.  Face Anti-Spoofing Based on Multi-layer Domain Adaptation , 2019, 2019 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[26]  Gerald Tesauro,et al.  Learning to Learn without Forgetting By Maximizing Transfer and Minimizing Interference , 2018, ICLR.

[27]  Stefanos Zafeiriou,et al.  ArcFace: Additive Angular Margin Loss for Deep Face Recognition , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Xilin Chen,et al.  Cross-Domain Face Presentation Attack Detection via Multi-Domain Disentangled Representation Learning , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  William J. Christmas,et al.  An anomaly detection approach to face spoofing detection: A new formulation and evaluation protocol , 2017, 2017 IEEE International Joint Conference on Biometrics (IJCB).

[30]  Xilin Chen,et al.  Unsupervised Adversarial Domain Adaptation for Cross-Domain Face Presentation Attack Detection , 2021, IEEE Transactions on Information Forensics and Security.

[31]  Xiangyu Zhu,et al.  Deep Spatial Gradient and Temporal Depth Learning for Face Anti-Spoofing , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Xiaoming Liu,et al.  Learning Deep Models for Face Anti-Spoofing: Binary or Auxiliary Supervision , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[33]  Xin Yu,et al.  Progressive Transfer Learning for Face Anti-Spoofing , 2021, IEEE Transactions on Image Processing.

[34]  Siome Goldenstein,et al.  Leveraging Shape, Reflectance and Albedo From Shading for Face Presentation Attack Detection , 2020, IEEE Transactions on Information Forensics and Security.

[35]  Xiaoming Liu,et al.  Face anti-spoofing using patch and depth-based CNNs , 2017, 2017 IEEE International Joint Conference on Biometrics (IJCB).

[36]  Anil K. Jain,et al.  Secure Face Unlock: Spoof Detection on Smartphones , 2016, IEEE Transactions on Information Forensics and Security.

[37]  Mengyang Liu,et al.  Deep learning for face anti-spoofing: An end-to-end approach , 2017, 2017 Signal Processing: Algorithms, Architectures, Arrangements, and Applications (SPA).

[38]  Anjith George,et al.  Deep Pixel-wise Binary Supervision for Face Presentation Attack Detection , 2019, 2019 International Conference on Biometrics (ICB).

[39]  Luc Van Gool,et al.  Domain Agnostic Feature Learning for Image and Video Based Face Anti-spoofing , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[40]  Xi Zhou,et al.  Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network , 2018, ECCV.

[41]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[42]  Yu Qiao,et al.  A Discriminative Feature Learning Approach for Deep Face Recognition , 2016, ECCV.

[43]  Stan Z. Li,et al.  Learn Convolutional Neural Network for Face Anti-Spoofing , 2014, ArXiv.

[44]  Guoying Zhao,et al.  Deep Learning for Face Anti-Spoofing: A Survey , 2021, ArXiv.

[45]  Yoshua Bengio,et al.  Semi-supervised Learning by Entropy Minimization , 2004, CAP.

[46]  Roberto J. López-Sastre,et al.  Learning to Learn Face-PAD: a lifelong learning approach , 2020, 2020 IEEE International Joint Conference on Biometrics (IJCB).

[47]  Pieter Abbeel,et al.  InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets , 2016, NIPS.

[48]  Weihong Deng,et al.  Learning temporal features using LSTM-CNN architecture for face anti-spoofing , 2015, 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR).

[49]  Shiguang Shan,et al.  Unified unsupervised and semi-supervised domain adaptation network for cross-scenario face anti-spoofing , 2021, Pattern Recognit..

[50]  Eduardo Valle,et al.  Transfer Learning Using Convolutional Neural Networks for Face Anti-spoofing , 2017, ICIAR.

[51]  Anderson Rocha,et al.  Face liveness detection under bad illumination conditions , 2011, 2011 18th IEEE International Conference on Image Processing.

[52]  Yuan Xie,et al.  Dual Reweighting Domain Generalization for Face Presentation Attack Detection , 2021, IJCAI.

[53]  Victor S. Lempitsky,et al.  Unsupervised Domain Adaptation by Backpropagation , 2014, ICML.

[54]  Wenhan Luo,et al.  Face Anti-Spoofing: Model Matters, so Does Data , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[55]  Muriel Visani,et al.  A Survey on Anti-Spoofing Methods for Facial Recognition with RGB Cameras of Generic Consumer Devices , 2020, J. Imaging.

[56]  Di Tang,et al.  Face Flashing: a Secure Liveness Detection Protocol based on Light Reflections , 2018, NDSS.

[57]  Jacques Wainer,et al.  Detecting face presentation attacks in mobile devices with a patch-based CNN and a sensor-aware loss function , 2020, PloS one.

[58]  D. Rus,et al.  Deep Evidential Regression , 2019, NeurIPS.

[59]  Xing Ji,et al.  CosFace: Large Margin Cosine Loss for Deep Face Recognition , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[60]  Seong-Whan Lee,et al.  Domain Generalization with Pseudo-Domain Label for Face Anti-Spoofing , 2021, ACPR.

[61]  Hamid R. Arabnia,et al.  A Brief Review of Domain Adaptation , 2020, Advances in Data Science and Information Engineering.

[62]  Jiwen Dong,et al.  shallowCNN-LE: A shallow CNN with Laplacian Embedding for face anti-spoofing , 2019, 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019).

[63]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[64]  Junjie Yan,et al.  A face antispoofing database with diverse attacks , 2012, 2012 5th IAPR International Conference on Biometrics (ICB).

[65]  Yaowu Chen,et al.  Attention-Based Two-Stream Convolutional Networks for Face Spoofing Detection , 2020, IEEE Transactions on Information Forensics and Security.

[66]  Sébastien Marcel,et al.  Domain Adaptation for Generalization of Face Presentation Attack Detection in Mobile Settengs with Minimal Information , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[67]  Yonghyun Kim,et al.  Suppressing Spoof-Irrelevant Factors for Domain-Agnostic Face Anti-Spoofing , 2020, IEEE Access.

[68]  Xilin Chen,et al.  Improving Cross-database Face Presentation Attack Detection via Adversarial Domain Adaptation , 2019, 2019 International Conference on Biometrics (ICB).

[69]  Yidong Li,et al.  CelebA-Spoof: Large-Scale Face Anti-Spoofing Dataset with Rich Annotations , 2020, ECCV.

[70]  Junying Gan,et al.  3D Convolutional Neural Network Based on Face Anti-spoofing , 2017, 2017 2nd International Conference on Multimedia and Image Processing (ICMIP).

[71]  Ville Ojansivu,et al.  Blur Insensitive Texture Classification Using Local Phase Quantization , 2008, ICISP.

[72]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[73]  Tieniu Tan,et al.  A Light CNN for Deep Face Representation With Noisy Labels , 2015, IEEE Transactions on Information Forensics and Security.

[74]  Mei Xie,et al.  Deep transfer across domains for face antispoofing , 2019, J. Electronic Imaging.

[75]  Roberto Javier López-Sastre,et al.  Generalized Presentation Attack Detection: a face anti-spoofing evaluation proposal , 2019, 2019 International Conference on Biometrics (ICB).

[76]  Sébastien Marcel,et al.  The Replay-Mobile Face Presentation-Attack Database , 2016, 2016 International Conference of the Biometrics Special Interest Group (BIOSIG).

[77]  Ying Huang,et al.  Deep Frequent Spatial Temporal Learning for Face Anti-Spoofing , 2020, ArXiv.

[78]  Guoying Zhao,et al.  3D Mask Face Anti-spoofing with Remote Photoplethysmography , 2016, ECCV.

[79]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[80]  Sébastien Marcel,et al.  Motion-based counter-measures to photo attacks in face recognition , 2014, IET Biom..

[81]  Feiyue Huang,et al.  Unsupervised Domain Adaptation for Face Anti-Spoofing , 2018, IEEE Transactions on Information Forensics and Security.

[82]  Jiaying Liu,et al.  Demystifying Neural Style Transfer , 2017, IJCAI.

[83]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[84]  Vighnesh Birodkar,et al.  Unsupervised Learning of Disentangled Representations from Video , 2017, NIPS.

[85]  Ramesh Raskar,et al.  Pairwise Confusion for Fine-Grained Visual Classification , 2017, ECCV.

[86]  Ashok Veeraraghavan,et al.  PPGSecure: Biometric Presentation Attack Detection Using Photopletysmograms , 2017, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[87]  Abdenour Hadid,et al.  Face Anti-spoofing using Hybrid Residual Learning Framework , 2019, 2019 International Conference on Biometrics (ICB).

[88]  Sergio Escalera,et al.  Contrastive Context-Aware Learning for 3D High-Fidelity Mask Face Presentation Attack Detection , 2021, IEEE Transactions on Information Forensics and Security.

[89]  Murat Sensoy,et al.  Evidential Deep Learning to Quantify Classification Uncertainty , 2018, NeurIPS.

[90]  Xiangyu Zhu,et al.  Learning Meta Model for Zero- and Few-Shot Face Anti-Spoofing , 2019, AAAI.

[91]  Marina Vannucci,et al.  Bayesian statistics and modelling , 2020, Nature Reviews Methods Primers.

[92]  Pong C. Yuen,et al.  Joint Discriminative Learning of Deep Dynamic Textures for 3D Mask Face Anti-Spoofing , 2019, IEEE Transactions on Information Forensics and Security.

[93]  Luc Van Gool,et al.  Unsupervised Compound Domain Adaptation for Face Anti-Spoofing , 2021, 2021 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021).

[94]  Yann LeCun,et al.  Dimensionality Reduction by Learning an Invariant Mapping , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[95]  Feiyue Huang,et al.  Generalizable Representation Learning for Mixture Domain Face Anti-Spoofing , 2021, AAAI.

[96]  Tatsuya Harada,et al.  Domain Generalization Using a Mixture of Multiple Latent Domains , 2019, AAAI.

[97]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[98]  Jukka Komulainen,et al.  OULU-NPU: A Mobile Face Presentation Attack Database with Real-World Variations , 2017, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[99]  Xiaohan Chen,et al.  Can We Gain More from Orthogonality Regularizations in Training Deep CNNs? , 2018, NeurIPS.

[100]  Jingang Shi,et al.  Face Anti-Spoofing with Human Material Perception , 2020, ECCV.

[101]  Alexander Binder,et al.  Deep One-Class Classification , 2018, ICML.

[102]  Roberto Javier López-Sastre,et al.  Deep Anomaly Detection for Generalized Face Anti-Spoofing , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[103]  Ying He,et al.  Retrieval-Based Face Annotation by Weak Label Regularized Local Coordinate Coding , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[104]  Pong C. Yuen,et al.  Regularized Fine-grained Meta Face Anti-spoofing , 2019, AAAI.

[105]  Adina Magda Florea,et al.  Comparison between Recurrent Networks and Temporal Convolutional Networks Approaches for Skeleton-Based Action Recognition , 2021, Sensors.

[106]  Xilin Chen,et al.  Single-Side Domain Generalization for Face Anti-Spoofing , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[107]  Jeffrey Scott Vitter,et al.  Random sampling with a reservoir , 1985, TOMS.

[108]  Chenxu Zhao,et al.  Searching Central Difference Convolutional Networks for Face Anti-Spoofing , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[109]  Yonghyun Kim,et al.  BASN: Enriching Feature Representation Using Bipartite Auxiliary Supervisions for Face Anti-Spoofing , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[110]  Yi Li,et al.  Face Liveness Detection from a Single Image with Sparse Low Rank Bilinear Discriminative Model , 2010, ECCV.

[111]  Ausif Mahmood,et al.  Deep face liveness detection based on nonlinear diffusion using convolution neural network , 2016, Signal, Image and Video Processing.

[112]  Signal Processing: Algorithms, Architectures, Arrangements, and Applications, SPA 2019, Poznan, Poland, September 18-20, 2019 , 2015, SPA.

[113]  LinLin Shen,et al.  Fusing Multiple Deep Features for Face Anti-spoofing , 2018, CCBR.

[114]  Jun Wan,et al.  NAS-FAS: Static-Dynamic Central Difference Network Search for Face Anti-Spoofing , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[115]  Sébastien Marcel,et al.  Spoofing Face Recognition With 3D Masks , 2014, IEEE Transactions on Information Forensics and Security.

[116]  Xiaoming Liu,et al.  On Disentangling Spoof Trace for Generic Face Anti-Spoofing , 2020, ECCV.

[117]  Anil K. Jain,et al.  Face Spoof Detection With Image Distortion Analysis , 2015, IEEE Transactions on Information Forensics and Security.

[118]  Shiqi Wang,et al.  Face Anti-Spoofing With Deep Neural Network Distillation , 2020, IEEE Journal of Selected Topics in Signal Processing.

[119]  Matti Pietikäinen,et al.  Performance evaluation of texture measures with classification based on Kullback discrimination of distributions , 1994, Proceedings of 12th International Conference on Pattern Recognition.