A survey on 3D mask presentation attack detection and countermeasures

Abstract Despite the impressive progress in face recognition, current systems are vulnerable to presentation attacks, which subvert the face recognition systems by presenting a face artifact. Several techniques have been developed to automatically detect different presentation attacks, mostly for 2D photo print and video replay attacks. However, with the development of 3D modeling and printing technologies, 3D mask has become a more effective way to attack the face recognition systems. Over the last decade, various detection methods for 3D mask attacks have been proposed, but there is no survey yet to summarize the advances. We present a comprehensive overview of the state-of-the-art approaches in 3D mask spoofing and anti-spoofing, including existing databases and countermeasures. In addition, we quantitatively compare the performance of different mask spoofing detection methods on a common ground (i.e., using the same database and evaluation metric). The effectiveness of several 2D presentation attack detection methods is also evaluated on two 3D mask spoofing databases to show whether they are applicable or not for 3D mask attacks. Finally, we present some insights and summarize open issues to address in the future.

[1]  Ramachandra Raghavendra,et al.  Novel presentation attack detection algorithm for face recognition system: Application to 3D face mask attack , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[2]  Fei Peng,et al.  Face presentation attack detection using guided scale texture , 2017, Multimedia Tools and Applications.

[3]  Sébastien Marcel,et al.  Face Recognition Systems Under Spoofing Attacks , 2016, Face Recognition Across the Imaging Spectrum.

[4]  Matti Pietikäinen,et al.  Generalized face anti-spoofing by detecting pulse from face videos , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[5]  Sébastien Marcel,et al.  What You Can't See Can Help You - Extended-Range Imaging for 3D-Mask Presentation Attack Detection , 2017, 2017 International Conference of the Biometrics Special Interest Group (BIOSIG).

[6]  Jean-Luc Dugelay,et al.  Shape and Texture Based Countermeasure to Protect Face Recognition Systems against Mask Attacks , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[7]  Abdenour Hadid,et al.  Face recognition under spoofing attacks: countermeasures and research directions , 2018, IET Biom..

[8]  Sébastien Marcel,et al.  Counter-measures to photo attacks in face recognition: A public database and a baseline , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[9]  Jukka Komulainen,et al.  Face Antispoofing Using Speeded-Up Robust Features and Fisher Vector Encoding , 2017, IEEE Signal Processing Letters.

[10]  Q. M. Jonathan Wu,et al.  A survey of local feature methods for 3D face recognition , 2017, Pattern Recognit..

[11]  Josef Bigün,et al.  Verifying liveness by multiple experts in face biometrics , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[12]  Farzin Deravi,et al.  Biometrie presentation attack detection using gaze alignment , 2018, 2018 IEEE 4th International Conference on Identity, Security, and Behavior Analysis (ISBA).

[13]  David Windridge,et al.  Detection of Face Spoofing Using Visual Dynamics , 2015, IEEE Transactions on Information Forensics and Security.

[14]  Olegs Nikisins,et al.  Biometric Face Presentation Attack Detection With Multi-Channel Convolutional Neural Network , 2019, IEEE Transactions on Information Forensics and Security.

[15]  Richa Singh,et al.  Face anti-spoofing using Haralick features , 2016, 2016 IEEE 8th International Conference on Biometrics Theory, Applications and Systems (BTAS).

[16]  Keche Mokhtar,et al.  A self-immune to 3D masks attacks face recognition system , 2018, Signal Image Video Process..

[17]  Feiyue Huang,et al.  Unsupervised Domain Adaptation for Face Anti-Spoofing , 2018, IEEE Transactions on Information Forensics and Security.

[18]  Pong C. Yuen,et al.  Joint Discriminative Learning of Deep Dynamic Textures for 3D Mask Face Anti-Spoofing , 2019, IEEE Transactions on Information Forensics and Security.

[19]  Samarth Bharadwaj,et al.  Face anti-spoofing with multifeature videolet aggregation , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[20]  Pong C. Yuen,et al.  Remote Photoplethysmography Correspondence Feature for 3D Mask Face Presentation Attack Detection , 2018, ECCV.

[21]  Davis E. King,et al.  Dlib-ml: A Machine Learning Toolkit , 2009, J. Mach. Learn. Res..

[22]  Farzin Deravi,et al.  Biometric Counter-Spoofing for Mobile Devices Using Gaze Information , 2017, PReMI.

[23]  Stan Z. Li,et al.  Face liveness detection by learning multispectral reflectance distributions , 2011, Face and Gesture 2011.

[24]  Liming Chen,et al.  3D Facial Geometric Attributes Based Anti-Spoofing Approach against Mask Attacks , 2017, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[25]  Sébastien Marcel,et al.  Spoofing Face Recognition With 3D Masks , 2014, IEEE Transactions on Information Forensics and Security.

[26]  Sébastien Marcel,et al.  Biometric Antispoofing Methods: A Survey in Face Recognition , 2014, IEEE Access.

[27]  David Menotti,et al.  Deep Representations for Iris, Face, and Fingerprint Spoofing Detection , 2014, IEEE Transactions on Information Forensics and Security.

[28]  Jian-Huang Lai,et al.  WLD-TOP Based Algorithm against Face Spoofing Attacks , 2015, CCBR.

[29]  Sung Wook Baik,et al.  CNN-based anti-spoofing two-tier multi-factor authentication system , 2019, Pattern Recognit. Lett..

[30]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[31]  Liming Chen,et al.  Shape Analysis Based Anti-spoofing 3D Face Recognition with Mask Attacks , 2016, RFMI.

[32]  Jean-Luc Dugelay,et al.  Countermeasure for the protection of face recognition systems against mask attacks , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[33]  S Naveen,et al.  Face recognition and authentication using LBP and BSIF mask detection and elimination , 2016, 2016 International Conference on Communication Systems and Networks (ComNet).

[34]  Ramachandra Raghavendra,et al.  Presentation Attack Detection Methods for Face Recognition Systems , 2017, ACM Comput. Surv..

[35]  Andreas Kolb,et al.  Reliable face anti-spoofing using multispectral SWIR imaging , 2016, 2016 International Conference on Biometrics (ICB).

[36]  Xu Zhao,et al.  Discriminative Representation Combinations for Accurate Face Spoofing Detection , 2018, Pattern Recognit..

[37]  Sakiko Yoshikawa,et al.  Hyper-realistic face masks: a new challenge in person identification , 2017, Cognitive Research: Principles and Implications.

[38]  Jean-Luc Dugelay,et al.  Reflectance analysis based countermeasure technique to detect face mask attacks , 2013, 2013 18th International Conference on Digital Signal Processing (DSP).

[39]  Sébastien Marcel,et al.  Spoofing in 2D face recognition with 3D masks and anti-spoofing with Kinect , 2013, 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[40]  Jukka Komulainen,et al.  Face anti-spoofing based on color texture analysis , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[41]  Sanjay Kumar Singh,et al.  Fusion of electrocardiogram with unobtrusive biometrics: An efficient individual authentication system , 2012, Pattern Recognit. Lett..

[42]  Javier Galbally,et al.  Three-dimensional and two-and-a-half-dimensional face recognition spoofing using three-dimensional printed models , 2016, IET Biom..

[43]  Yuchun Fang,et al.  Ultra-deep Neural Network for Face Anti-spoofing , 2017, ICONIP.

[44]  Abdenour Hadid,et al.  On the use of Kinect depth data for identity, gender and ethnicity classification from facial images , 2015, Pattern Recognit. Lett..

[45]  Guoying Zhao,et al.  A 3D Mask Face Anti-Spoofing Database with Real World Variations , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[46]  Keche Mokhtar,et al.  The detection of spoofing by 3D mask in a 2D identity recognition system , 2017, Egyptian Informatics Journal.

[47]  Richa Singh,et al.  Detecting Silicone Mask-Based Presentation Attack via Deep Dictionary Learning , 2017, IEEE Transactions on Information Forensics and Security.

[48]  Ramachandra Raghavendra,et al.  Robust 2D/3D face mask presentation attack detection scheme by exploring multiple features and comparison score level fusion , 2014, 17th International Conference on Information Fusion (FUSION).

[49]  Pong C. Yuen,et al.  Deep convolutional dynamic texture learning with adaptive channel-discriminability for 3D mask face anti-spoofing , 2017, 2017 IEEE International Joint Conference on Biometrics (IJCB).

[50]  Javier Hernandez-Ortega,et al.  Time Analysis of Pulse-Based Face Anti-Spoofing in Visible and NIR , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[51]  Julian Fiérrez,et al.  Efficient software attack to multimodal biometric systems and its application to face and iris fusion , 2014, Pattern Recognit. Lett..

[52]  Junjie Yan,et al.  A face antispoofing database with diverse attacks , 2012, 2012 5th IAPR International Conference on Biometrics (ICB).

[53]  Fei Peng,et al.  A competition on generalized software-based face presentation attack detection in mobile scenarios , 2017, 2017 IEEE International Joint Conference on Biometrics (IJCB).

[54]  João Paulo Papa,et al.  How far did we get in face spoofing detection? , 2018, Eng. Appl. Artif. Intell..

[55]  Richa Singh,et al.  Face Presentation Attack with Latex Masks in Multispectral Videos , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[56]  Seongbeak Yoon,et al.  Masked fake face detection using radiance measurements. , 2009, Journal of the Optical Society of America. A, Optics, image science, and vision.

[57]  Xiaoli Hao,et al.  A New Multispectral Method for Face Liveness Detection , 2013, 2013 2nd IAPR Asian Conference on Pattern Recognition.

[58]  Jean-Luc Dugelay,et al.  Mask spoofing in face recognition and countermeasures , 2014, Image Vis. Comput..

[59]  Daijin Kim,et al.  Face spoofing detection with highlight removal effect and distortions , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[60]  Eduardo Valle,et al.  Transfer Learning Using Convolutional Neural Networks for Face Anti-spoofing , 2017, ICIAR.

[61]  Anil K. Jain,et al.  Face Spoof Detection With Image Distortion Analysis , 2015, IEEE Transactions on Information Forensics and Security.

[62]  Farzin Deravi,et al.  Gaze stability for liveness detection , 2016, Pattern Analysis and Applications.

[63]  Anderson Rocha,et al.  Face Spoofing Detection Through Visual Codebooks of Spectral Temporal Cubes , 2015, IEEE Transactions on Image Processing.

[64]  Sushil K. Bhattacharjee,et al.  Recent Advances in Face Presentation Attack Detection , 2019, Handbook of Biometric Anti-Spoofing, 2nd Ed..

[65]  Lai-Man Po,et al.  Integration of image quality and motion cues for face anti-spoofing: A neural network approach , 2016, J. Vis. Commun. Image Represent..

[66]  Ajay Kumar,et al.  Detecting Presentation Attacks from 3D Face Masks Under Multispectral Imaging , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[67]  Matti Pietikäinen,et al.  Learning Discriminant Face Descriptor , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[68]  Matti Pietikäinen,et al.  Face spoofing detection from single images using micro-texture analysis , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[69]  Yan Wang,et al.  Face Anti-spoofing to 3D Masks by Combining Texture and Geometry Features , 2018, CCBR.

[70]  Jean-Luc Dugelay,et al.  On the vulnerability of face recognition systems to spoofing mask attacks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[71]  Petru Radu,et al.  Robust multimodal face and fingerprint fusion in the presence of spoofing attacks , 2016, Pattern Recognit..