Enhanced Deep Learning Architectures for Face Liveness Detection for Static and Video Sequences

Face liveness detection is a critical preprocessing step in face recognition for avoiding face spoofing attacks, where an impostor can impersonate a valid user for authentication. While considerable research has been recently done in improving the accuracy of face liveness detection, the best current approaches use a two-step process of first applying non-linear anisotropic diffusion to the incoming image and then using a deep network for final liveness decision. Such an approach is not viable for real-time face liveness detection. We develop two end-to-end real-time solutions where nonlinear anisotropic diffusion based on an additive operator splitting scheme is first applied to an incoming static image, which enhances the edges and surface texture, and preserves the boundary locations in the real image. The diffused image is then forwarded to a pre-trained Specialized Convolutional Neural Network (SCNN) and the Inception network version 4, which identify the complex and deep features for face liveness classification. We evaluate the performance of our integrated approach using the SCNN and Inception v4 on the Replay-Attack dataset and Replay-Mobile dataset. The entire architecture is created in such a manner that, once trained, the face liveness detection can be accomplished in real-time. We achieve promising results of 96.03% and 96.21% face liveness detection accuracy with the SCNN, and 94.77% and 95.53% accuracy with the Inception v4, on the Replay-Attack, and Replay-Mobile datasets, respectively. We also develop a novel deep architecture for face liveness detection on video frames that uses the diffusion of images followed by a deep Convolutional Neural Network (CNN) and a Long Short-Term Memory (LSTM) to classify the video sequence as real or fake. Even though the use of CNN followed by LSTM is not new, combining it with diffusion (that has proven to be the best approach for single image liveness detection) is novel. Performance evaluation of our architecture on the REPLAY-ATTACK dataset gave 98.71% test accuracy and 2.77% Half Total Error Rate (HTER), and on the REPLAY-MOBILE dataset gave 95.41% accuracy and 5.28% HTER.

[1]  Marsyita Hanafi,et al.  Face Liveness Detection Using Dynamic Local Ternary Pattern (DLTP) , 2016, Comput..

[2]  Marie-Francine Moens,et al.  A survey on the application of recurrent neural networks to statistical language modeling , 2015, Comput. Speech Lang..

[3]  Dongyu Zhang,et al.  Image-to-Video Person Re-Identification With Temporally Memorized Similarity Learning , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[4]  Ausif Mahmood,et al.  Deep face liveness detection based on nonlinear diffusion using convolution neural network , 2016, Signal, Image and Video Processing.

[5]  Wonjun Kim,et al.  Face liveness detection from a single image via diffusion speed model. , 2015, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[6]  Shengcai Liao,et al.  Face liveness detection using 3D structure recovered from a single camera , 2013, 2013 International Conference on Biometrics (ICB).

[7]  Samarth Bharadwaj,et al.  Computationally Efficient Face Spoofing Detection with Motion Magnification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[8]  Erkut Erdem,et al.  Linear Diffusion , 2022 .

[9]  Sébastien Marcel,et al.  LBP - TOP Based Countermeasure against Face Spoofing Attacks , 2012, ACCV Workshops.

[10]  Xinghao Jiang,et al.  3D Gait Recognition Based on a CNN-LSTM Network with the Fusion of SkeGEI and DA Features , 2019, 2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[11]  Anil K. Jain,et al.  Face Spoof Detection With Image Distortion Analysis , 2015, IEEE Transactions on Information Forensics and Security.

[12]  Wonjun Kim,et al.  Face Liveness Detection From a Single Image via Diffusion Speed Model , 2015, IEEE Transactions on Image Processing.

[13]  Sung Wook Baik,et al.  Action Recognition in Video Sequences using Deep Bi-Directional LSTM With CNN Features , 2018, IEEE Access.

[14]  Ausif Mahmood,et al.  Optimizing Deep CNN Architectures for Face Liveness Detection , 2019, Entropy.

[15]  Farzin Deravi,et al.  Spatio-Temporal Texture Features for Presentation Attack Detection in Biometric Systems , 2019, 2019 Eighth International Conference on Emerging Security Technologies (EST).

[16]  Ran He,et al.  Person identification from lip texture analysis , 2016, 2016 IEEE International Conference on Digital Signal Processing (DSP).

[17]  Muhammad Awais,et al.  Spoofing Attack Detection by Anomaly Detection , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[18]  Sébastien Marcel,et al.  Motion-based counter-measures to photo attacks in face recognition , 2014, IET Biom..

[19]  Sébastien Marcel,et al.  Counter-measures to photo attacks in face recognition: A public database and a baseline , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[20]  Mei Xie,et al.  Enhance the Motion Cues for Face Anti-Spoofing using CNN-LSTM Architecture , 2019, ArXiv.

[21]  Di Tang,et al.  Face Flashing: a Secure Liveness Detection Protocol based on Light Reflections , 2018, NDSS.

[22]  Luisa Verdoliva,et al.  Biometric Spoofing Detection by a Domain-Aware Convolutional Neural Network , 2016, 2016 12th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS).

[23]  Mengyang Liu,et al.  LiveNet: Improving features generalization for face liveness detection using convolution neural networks , 2018, Expert Syst. Appl..

[24]  Chun-Hsiao Yeh,et al.  Face Liveness Detection Based on Perceptual Image Quality Assessment Features with Multi-scale Analysis , 2018, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[25]  Chang Liu,et al.  A New Approach for Face Anti-Spoofing Using Handcrafted and Deep Network Features , 2019, 2019 IEEE International Conference on Service Operations and Logistics, and Informatics (SOLI).

[26]  Jarno Ralli.  PDE Based Image Diffusion and AOS , 2012 .

[27]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[28]  Sébastien Marcel,et al.  On the effectiveness of local binary patterns in face anti-spoofing , 2012, 2012 BIOSIG - Proceedings of the International Conference of Biometrics Special Interest Group (BIOSIG).

[29]  M. Omair Ahmad,et al.  An Edge-Adapting Laplacian Kernel For Nonlinear Diffusion Filters , 2012, IEEE Transactions on Image Processing.

[30]  Weihong Deng,et al.  Learning temporal features using LSTM-CNN architecture for face anti-spoofing , 2015, 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR).

[31]  Sébastien Marcel,et al.  The Replay-Mobile Face Presentation-Attack Database , 2016, 2016 International Conference of the Biometrics Special Interest Group (BIOSIG).

[32]  Sébastien Marcel,et al.  On Effectiveness of Anomaly Detection Approaches against Unseen Presentation Attacks in Face Anti-spoofing , 2018, 2018 International Conference on Biometrics (ICB).

[33]  Sébastien Marcel,et al.  Image Quality Assessment for Fake Biometric Detection: Application to Iris, Fingerprint, and Face Recognition , 2014, IEEE Transactions on Image Processing.

[34]  Shervin Rahimzadeh Arashloo.  Unseen Face Presentation Attack Detection Using Class-Specific Sparse One-Class Multiple Kernel Fusion Regression , 2019, ArXiv.

[35]  Zhu Ming,et al.  CNN based spatio-temporal feature extraction for face anti-spoofing , 2017, 2017 2nd International Conference on Image, Vision and Computing (ICIVC).

[36]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Ausif Mahmood,et al.  Enhancing computer vision to detect face spoofing attack utilizing a single frame from a replay video attack using deep learning , 2016, 2016 International Conference on Optoelectronics and Image Processing (ICOIP).

[38]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[39]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  Max A. Viergever,et al.  Efficient and reliable schemes for nonlinear diffusion filtering , 1998, IEEE Trans. Image Process..