Differential 3D Facial Recognition: Adding 3D to Your State-of-the-Art 2D Method

Active illumination is a prominent complement to enhance 2D face recognition and make it more robust, e.g., to spoofing attacks and low-light conditions. In the present work we show that it is possible to adopt active illumination to enhance state-of-the-art 2D face recognition approaches with 3D features, while bypassing the complicated task of 3D reconstruction. The key idea is to project over the test face a high spatial frequency pattern, which allows us to simultaneously recover real 3D information plus a standard 2D facial image. Therefore, state-of-the-art 2D face recognition solution can be transparently applied, while from the high frequency component of the input image, complementary 3D facial features are extracted. Experimental results on ND-2006 dataset show that the proposed ideas can significantly boost face recognition performance and dramatically improve the robustness to spoofing attacks.

[1]  Xiaoming Liu,et al.  Learning Deep Models for Face Anti-Spoofing: Binary or Auxiliary Supervision , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[2]  Song Zhang,et al.  High-resolution, real-time 3D absolute coordinate measurement based on a phase-shifting method. , 2006, Optics express.

[3]  Stefanos Zafeiriou,et al.  ArcFace: Additive Angular Margin Loss for Deep Face Recognition , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Josef Kittler,et al.  Face Recognition Using Active Near-IR Illumination , 2005, BMVC.

[5]  Yu Liu,et al.  Exploring Disentangled Feature Representation Beyond Face Identification , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[6]  Xing Ji,et al.  CosFace: Large Margin Cosine Loss for Deep Face Recognition , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[7]  Rama Chellappa,et al.  Enforcing integrability by error correction using l1-minimization , 2009, CVPR.

[8]  Junjie Yan,et al.  A face antispoofing database with diverse attacks , 2012, 2012 5th IAPR International Conference on Biometrics (ICB).

[9]  Y. Kaya,et al.  A BASIC STUDY ON HUMAN FACE RECOGNITION , 1972 .

[10]  Marios Savvides,et al.  Ring Loss: Convex Feature Normalization for Face Recognition , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[11]  Amnon Shashua,et al.  On Photometric Issues in 3D Visual Recognition from a Single 2D Image , 2004, International Journal of Computer Vision.

[12]  L. Schwartz Théorie des distributions , 1966 .

[13]  Fang Zhao,et al.  Towards Pose Invariant Face Recognition in the Wild , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[14]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Jukka Komulainen,et al.  OULU-NPU: A Mobile Face Presentation Attack Database with Real-World Variations , 2017, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[16]  Yajun Wang,et al.  Some recent advances on superfast 3D shape measurement with digital binary defocusing techniques , 2014 .

[17]  Syed Zulqarnain Gilani,et al.  Learning from Millions of 3D Scans for Large-Scale 3D Face Recognition , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[18]  Andrew Zisserman,et al.  Deep Face Recognition , 2015, BMVC.

[19]  Davis E. King,et al.  Dlib-ml: A Machine Learning Toolkit , 2009, J. Mach. Learn. Res..

[20]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[21]  Petra Kaufmann,et al.  Two Dimensional Phase Unwrapping Theory Algorithms And Software , 2016 .

[22]  Song Zhang Recent progresses on real-time 3D shape measurement using digital fringe projection techniques , 2010 .

[23]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Zhenan Sun,et al.  Dynamic Feature Learning for Partial Face Recognition , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[25]  Guosheng Lin,et al.  Deep convolutional neural fields for depth estimation from a single image , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Bhiksha Raj,et al.  SphereFace: Deep Hypersphere Embedding for Face Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Chun-Hsiao Yeh,et al.  Face liveness detection with feature discrimination between sharpness and blurriness , 2017, 2017 Fifteenth IAPR International Conference on Machine Vision Applications (MVA).

[28]  John W. Fisher,et al.  Information-Driven Adaptive Structured-Light Scanners , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  José A. Ferrari,et al.  One-shot 3D gradient field scanning , 2015 .

[30]  Nassir Navab,et al.  Deeper Depth Prediction with Fully Convolutional Residual Networks , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[31]  Rob Fergus,et al.  Depth Map Prediction from a Single Image using a Multi-Scale Deep Network , 2014, NIPS.

[32]  Song Zhang,et al.  Handbook of 3D Machine Vision: Optical Metrology and Imaging , 2013 .

[33]  Stefanos Zafeiriou,et al.  UV-GAN: Adversarial Facial UV Map Completion for Pose-Invariant Face Recognition , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[34]  Ramesh Raskar,et al.  Why I Want a Gradient Camera , 2022 .

[35]  Rita Cucchiara,et al.  Learning to Generate Facial Depth Maps , 2018, 2018 International Conference on 3D Vision (3DV).

[36]  Dennis C. Ghiglia,et al.  Two-Dimensional Phase Unwrapping: Theory, Algorithms, and Software , 1998 .

[37]  Stefan Winkler,et al.  A data-driven approach to cleaning large face datasets , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[38]  Rama Chellappa,et al.  An algebraic approach to surface reconstruction from gradient fields , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[39]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[40]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Rama Chellappa,et al.  What Is the Range of Surface Reconstructions from a Gradient Field? , 2006, ECCV.

[43]  Olivier D. Faugeras,et al.  Shape From Shading , 2006, Handbook of Mathematical Models in Computer Vision.

[44]  Gastón A. Ayubi,et al.  Pulse-width modulation in defocused three-dimensional fringe projection. , 2010, Optics letters.

[45]  William J. Christmas,et al.  A Multiresolution 3D Morphable Face Model and Fitting Framework , 2016, VISIGRAPP.

[46]  Roland Göcke,et al.  Joint Registration and Representation Learning for Unconstrained Face Identification , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  Ioannis A. Kakadiaris,et al.  End-to-End 3D Face Reconstruction with Deep Neural Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Xiaoming Liu,et al.  Face anti-spoofing using patch and depth-based CNNs , 2017, 2017 IEEE International Joint Conference on Biometrics (IJCB).

[49]  Salah Karout,et al.  Two-dimensional phase unwrapping , 2007 .

[50]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[51]  Xu Zhang,et al.  Face spoofing detection based on 3D lighting environment analysis of image pair , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[52]  Shiguang Shan,et al.  Improving 2D Face Recognition via Discriminative Face Depth Estimation , 2018, 2018 International Conference on Biometrics (ICB).

[53]  Rama Chellappa,et al.  Appearance Characterization of Linear Lambertian Objects, Generalized Photometric Stereo, and Illumination-Invariant Face Recognition , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[54]  Zhouyu Fu,et al.  Robust Surface Reconstruction from Gradient Field Using the L1 Norm , 2007, 9th Biennial Conference of the Australian Pattern Recognition Society on Digital Image Computing Techniques and Applications (DICTA 2007).

[55]  Ronen Basri,et al.  Lambertian reflectance and linear subspaces , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[56]  Nikos Paragios,et al.  Handbook of Mathematical Models in Computer Vision , 2005 .

[57]  Ira Kemelmacher-Shlizerman,et al.  Level Playing Field for Million Scale Face Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[58]  L. Schwartz Théorie des distributions à valeurs vectorielles. I , 1957 .

[59]  José A. Ferrari,et al.  One-shot 3D scanning by combining sparse landmarks with dense gradient information , 2018, Optics and Lasers in Engineering.

[60]  K.W. Bowyer,et al.  Using a Multi-Instance Enrollment Representation to Improve 3D Face Recognition , 2007, 2007 First IEEE International Conference on Biometrics: Theory, Applications, and Systems.

[61]  Rama Chellappa,et al.  Disentangling 3D Pose in a Dendritic CNN for Unconstrained 2D Face Alignment , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[62]  Xiaoming Liu,et al.  Disentangled Representation Learning GAN for Pose-Invariant Face Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[63]  Dacheng Tao,et al.  A Comprehensive Survey on Pose-Invariant Face Recognition , 2015, ACM Trans. Intell. Syst. Technol..

[64]  Tieniu Tan,et al.  Live face detection based on the analysis of Fourier spectra , 2004, SPIE Defense + Commercial Sensing.

[65]  Shiqi Wang,et al.  Learning Generalized Deep Feature Representation for Face Anti-Spoofing , 2018, IEEE Transactions on Information Forensics and Security.

[66]  M. Takeda,et al.  Fourier transform profilometry for the automatic measurement of 3-D object shapes. , 1983, Applied optics.

[67]  A. Hadid,et al.  Face anti-spoofing via deep local binary patterns , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[68]  Cheng Li,et al.  Pose-Robust Face Recognition via Deep Residual Equivariant Mapping , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[69]  Guillermo Sapiro,et al.  Not Afraid of the Dark: NIR-VIS Face Recognition via Cross-Spectral Hallucination and Low-Rank Embedding , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[70]  Xin Yu,et al.  Hallucinating Very Low-Resolution Unaligned and Noisy Face Images by Transformative Discriminative Autoencoders , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[71]  Subhash C. Bagui,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2005, Technometrics.

[72]  Stefanos Zafeiriou,et al.  Face Recognition and Verification Using Photometric Stereo: The Photoface Database and a Comprehensive Evaluation , 2013, IEEE Transactions on Information Forensics and Security.

[73]  Rob Fergus,et al.  Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-scale Convolutional Architecture , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[74]  Ira Kemelmacher-Shlizerman,et al.  The MegaFace Benchmark: 1 Million Faces for Recognition at Scale , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).