Blending 2D and 3D Face Recognition

Over the last decade, performance of face recognition algorithms systematically improved. This is particularly impressive when considering very large or challenging datasets such as the FRGC v2 or Labelled Faces in the Wild . A better analysis of the structure of the facial texture and shape is one of the main reasons of improvement in recognition performance. Hybrid face recognition methods , combining holistic and feature-based approaches, also allowed to increase efficiency and robustness. Both photometric information and shape information allow to extract facial features which can be exploited for recognition. However, both sources, grey levels of image pixels and 3D data , are affected by several noise sources which may impair the recognition performance. One of the main difficulties in matching 3D faces is the detection and localization of distinctive and stable points in 3D scans. Moreover, the large amount of data (tens of thousands of points) to be processed make the direct one-to-one matching a very time-consuming process. On the other hand, matching algorithms based on the analysis of 2D data alone are very sensitive to variations in illumination, expression and pose. Algorithms, based on the face shape information alone, are instead relatively insensitive to these sources of noise. These mutually exclusive features of 2D- and 3D-based face recognition algorithm call for a cooperative scheme which may take advantage of the strengths of both, while coping for their weaknesses. We envisage many real and practical applications where 2D data can be used to improve 3D matching and vice versa. Towards this end, this chapter highlights both the advantages and disadvantages of 2D- and 3D-based face recognition algorithms . It also explores the advantages of blending 2D- and 3D data -based techniques, also proposing a novel approach for a fast and robust matching. Several experimental results, obtained from publicly available datasets, currently at the state of the art, demonstrate the effectiveness of the proposed approach.

[1]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[2]  Lawrence Sirovich,et al.  Application of the Karhunen-Loeve Procedure for the Characterization of Human Faces , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[4]  Marinella Cadoni,et al.  Iconic Methods for Multimodal Face Recognition: A Comparative Study , 2014, 2014 22nd International Conference on Pattern Recognition.

[5]  Baochang Zhang,et al.  Local Derivative Pattern Versus Local Binary Pattern: Face Recognition With High-Order Local Pattern Descriptor , 2010, IEEE Transactions on Image Processing.

[6]  Raimondo Schettini,et al.  Three-Dimensional Occlusion Detection and Restoration of Partially Occluded Faces , 2011, Journal of Mathematical Imaging and Vision.

[7]  Avinash C. Kak,et al.  PCA versus LDA , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Mohammed Bennamoun,et al.  An Efficient Multimodal 2D-3D Hybrid Approach to Automatic Face Recognition , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Meng Joo Er,et al.  Face recognition with radial basis function (RBF) neural networks , 2002, IEEE Trans. Neural Networks.

[10]  Patrick J. Flynn,et al.  Face Recognition Using 2D and 3D Facial Data , 2003 .

[11]  L. Akarun,et al.  A 3D Face Recognition System for Expression and Occlusion Invariance , 2008, 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems.

[12]  Ah Chung Tsoi,et al.  Face recognition: a convolutional neural-network approach , 1997, IEEE Trans. Neural Networks.

[13]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Konstantinos N. Plataniotis,et al.  Face recognition using LDA-based algorithms , 2003, IEEE Trans. Neural Networks.

[15]  Sridha Sridharan,et al.  Combined 2D/3D Face Recognition Using Log-Gabor Templates , 2006, 2006 IEEE International Conference on Video and Signal Based Surveillance.

[16]  Patrick J. Flynn,et al.  A Region Ensemble for 3-D Face Recognition , 2008, IEEE Transactions on Information Forensics and Security.

[17]  Wei-Yang Lin,et al.  3D Face Recognition Under Expression Variations using Similarity Metrics Fusion , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[18]  Liming Chen,et al.  Towards 3D Face Recognition in the Real: A Registration-Free Approach Using Fine-Grained Matching of 3D Keypoint Descriptors , 2014, International Journal of Computer Vision.

[19]  J. Cartoux,et al.  Face authentification or recognition by profile extraction from range images , 1989, [1989] Proceedings. Workshop on Interpretation of 3D Scenes.

[20]  Norbert Krüger,et al.  Face Recognition by Elastic Bunch Graph Matching , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Marian Stewart Bartlett,et al.  Face recognition by independent component analysis , 2002, IEEE Trans. Neural Networks.

[22]  Luuk J. Spreeuwers,et al.  Fast and Accurate 3D Face Recognition , 2011, International Journal of Computer Vision.

[23]  Rabia Jafri,et al.  A Survey of Face Recognition Techniques , 2009, J. Inf. Process. Syst..

[24]  Anil K. Jain,et al.  Matching 2.5D face scans to 3D models , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Evangelos E. Milios,et al.  Matching range images of human faces , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[27]  Patrick J. Flynn,et al.  Multiple Nose Region Matching for 3D Face Recognition under Varying Facial Expression , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Alex Pentland,et al.  Face recognition using eigenfaces , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[29]  Chengjun Liu,et al.  Gabor-based kernel PCA with fractional power polynomial models for face recognition , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Andrea F. Abate,et al.  2D and 3D face recognition: A survey , 2007, Pattern Recognit. Lett..

[31]  Yuxiao Hu,et al.  Learning a locality preserving subspace for visual recognition , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[32]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Peter J. Olver,et al.  Joint Invariant Signatures , 2001, Found. Comput. Math..

[34]  Sun-Yuan Kung,et al.  Face recognition/detection by probabilistic decision-based neural network , 1997, IEEE Trans. Neural Networks.

[35]  Patrick J. Flynn,et al.  A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition , 2006, Comput. Vis. Image Underst..

[36]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[37]  J. Hadamard,et al.  Lectures on Cauchy's Problem in Linear Partial Differential Equations , 1924 .

[38]  Amnon Shashua,et al.  Manifold pursuit: a new approach to appearance based recognition , 2002, Object recognition supported by user interaction for service robots.

[39]  Xiaoou Tang,et al.  Robust 3D Face Recognition by Local Shape Difference Boosting , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[41]  Hassen Drira,et al.  3D Face Recognition under Expressions, Occlusions, and Pose Variations , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Ioannis A. Kakadiaris,et al.  3D Face Discriminant Analysis Using Gauss-Markov Posterior Marginals , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Paul Suetens,et al.  meshSIFT: Local surface features for 3D face recognition under expression variations and partial data , 2013, Comput. Vis. Image Underst..

[44]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[45]  Andrea Lagorio,et al.  On the Use of SIFT Features for Face Authentication , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[46]  Ioannis A. Kakadiaris,et al.  Three-Dimensional Face Recognition in the Presence of Facial Expressions: An Annotated Deformable Model Approach , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Patrick J. Flynn,et al.  Adaptive Rigid Multi-region Selection for Handling Expression Variation in 3D Face Recognition , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[48]  Berk Gökberk,et al.  Regional Registration for Expression Resistant 3-D Face Recognition , 2010, IEEE Transactions on Information Forensics and Security.

[49]  Patrick J. Flynn,et al.  Impact of involuntary subject movement on 3D face scans , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[50]  Alberto Del Bimbo,et al.  3D Face Recognition Using Isogeodesic Stripes , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  Faisal R. Al-Osaimi,et al.  An Expression Deformation Approach to Non-rigid 3D Face Recognition , 2009, International Journal of Computer Vision.

[52]  Christoph von der Malsburg,et al.  Strategies and Benefits of Fusion of 2D and 3D Face Recognition , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[53]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[54]  Maurício Pamplona Segundo,et al.  3D Face Recognition Using Simulated Annealing and the Surface Interpenetration Measure , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  Ioannis A. Kakadiaris,et al.  UR3D-C: Linear dimensionality reduction for efficient 3D face recognition , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[56]  Alberto Del Bimbo,et al.  Matching 3D face scans using interest points and local histogram descriptors , 2013, Comput. Graph..

[57]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[58]  M. K. Fleming,et al.  Categorization of faces using unsupervised feature extraction , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[59]  Kap Luk Chan,et al.  Face recognition based on discriminative manifold learning , 2004, ICPR 2004.

[60]  Gérard G. Medioni,et al.  Performance of Geometrix ActiveID^TM 3D Face Recognition Engine on the FRGC Data , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.