Comparison of 2 D / 3 D Features and Their Adaptive Score Level Fusion for 3 D Face Recognition

3D face has been introduced in the literature to deal with the unsolved issues of 2D face recognition, namely lighting and pose variations. In this paper, we study and compare the distinctiveness of features extracted from both the registered 2D face images and 3D face models. Sparse Representation Classifier (SRC) is exploited to calculate all similarity measures which are compared with the ones by a baseline of Nearest Neighbor (NN). As individual features of 2D and 3D are far from distinctive for discriminating human faces, we further present an adaptive score level fusion strategy for multimodal 2D-3D face recognition. The novel fusion strategy consists of an offline and an online weight learning process, both of which automatically select the most relevant weights of all the scores for each probe face in each modality. The weights calculated offline are based on the EER value of each type of features, while the online ones are dynamically obtained according to matching scores. Both types of weights are then fused to generate a final weight. Tested on the complete FRGC v2.0 dataset, the best rank-one recognition rate using only 3D or 2D features is 79.72% and 77.89%, respectively; while the new proposed adaptive fusion strategy achieves 95.48% with a 97.03% verification rate at 0.001 FAR, highlighting the benefit of exploring both 3D and 2D clues as well as the effectiveness of our adaptive fusion strategy.

[1]  S Marcelja,et al.  Mathematical description of the responses of simple cortical cells. , 1980, Journal of the Optical Society of America.

[2]  Hiromi T. Tanaka,et al.  Curvature-based face surface recognition using spherical correlation-principal directions for curved object recognition , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[3]  Norbert Krüger,et al.  Face Recognition by Elastic Bunch Graph Matching , 1997, CAIP.

[4]  Shihong Lao,et al.  3D template matching for pose invariant face recognition using 3D facial model built with isoluminance line based stereo vision , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[5]  Marc Acheroy,et al.  Face verification from 3D and grey level clues , 2001, Pattern Recognit. Lett..

[6]  Chin-Seng Chua,et al.  Facial feature detection and face recognition from 2D and 3D images , 2002, Pattern Recognit. Lett..

[7]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[8]  Michael G. Strintzis,et al.  Use of depth and colour eigenfaces for face recognition , 2003, Pattern Recognit. Lett..

[9]  Afzal Godil,et al.  Face recognition using 3D facial shape and color map information: comparison and combination , 2004, SPIE Defense + Commercial Sensing.

[10]  Daniel Rueckert,et al.  Evaluation of automatic 4D face recognition using surface and texture registration , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[11]  Arun Ross,et al.  An introduction to biometric recognition , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Matti Pietikäinen,et al.  Face Recognition with Local Binary Patterns , 2004, ECCV.

[13]  Patrick J. Flynn,et al.  Overview of the face recognition grand challenge , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[14]  Christoph von der Malsburg,et al.  Strategies and Benefits of Fusion of 2D and 3D Face Recognition , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[15]  Anil K. Jain,et al.  Integrating Range and Texture Information for 3D Face Recognition , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[16]  Anil K. Jain,et al.  Large-scale evaluation of multimodal biometric authentication using state-of-the-art systems , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Berk Gökberk,et al.  Rank-based decision fusion for 3D shape-based face recognition , 2005, SIU 2005.

[18]  Berk Gökberk,et al.  Rank-based decision fusion for 3D shape-based face recognition , 2005, Proceedings of the IEEE 13th Signal Processing and Communications Applications Conference, 2005..

[19]  Remco C. Veltkamp,et al.  A Survey of 3D Face Recognition Methods , 2005, AVBPA.

[20]  Gérard G. Medioni,et al.  Performance of Geometrix ActiveID^TM 3D Face Recognition Engine on the FRGC Data , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[21]  Patrick J. Flynn,et al.  3D Face Recognition with Region Committee Voting , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[22]  S. Zucker,et al.  Differential Geometry from the Frenet Point of View: Boundary Detection, Stereo, Texture and Color , 2006, Handbook of Mathematical Models in Computer Vision.

[23]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[24]  Lale Akarun,et al.  Representation Plurality and Decision Level Fusion for 3D Face Recognition , 2006 .

[25]  Ke Huang,et al.  Sparse Representation for Signal Classification , 2006, NIPS.

[26]  Face Recognition Using 2D and 3D Multimodal Local Features , 2006, ISVC.

[27]  Patrick J. Flynn,et al.  A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition , 2006, Comput. Vis. Image Underst..

[28]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[29]  Berk Gökberk,et al.  Comparative Analysis of Decision-level Fusion Algorithms for 3D Face Recognition , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[30]  Liming Chen,et al.  New Experiments on ICP-Based 3D Face Recognition and Authentication , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[31]  Anil K. Jain,et al.  Matching 2.5D face scans to 3D models , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Mohammed Bennamoun,et al.  Keypoint Detection and Local Feature Matching for Textured 3D Face Recognition , 2007, International Journal of Computer Vision.

[33]  Mohammed Bennamoun,et al.  An Efficient Multimodal 2D-3D Hybrid Approach to Automatic Face Recognition , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Raffaella Lanzarotti,et al.  Face Recognition Based on 2D and 3D Features , 2007, KES.

[35]  Sridha Sridharan,et al.  Robust 3D Face Recognition from Expression Categorisation , 2007, ICB.

[36]  Ioannis A. Kakadiaris,et al.  Three-Dimensional Face Recognition in the Presence of Facial Expressions: An Annotated Deformable Model Approach , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Marina L. Gavrilova,et al.  FES: A System for Combining Face, Ear and Signature Biometrics Using Rank Level Fusion , 2008, Fifth International Conference on Information Technology: New Generations (itng 2008).

[38]  Patrick J. Flynn,et al.  Using multi-instance enrollment to improve performance of 3D face recognition , 2008, Comput. Vis. Image Underst..

[39]  Jiuchao Feng,et al.  KFCE: A dictionary generation algorithm for sparse representation , 2009, Signal Process..

[40]  Liming Chen,et al.  A coarse-to-fine curvature analysis-based rotation invariant 3D face landmarking , 2009, 2009 IEEE 3rd International Conference on Biometrics: Theory, Applications, and Systems.

[41]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Ammad Ali,et al.  Face Recognition with Local Binary Patterns , 2012 .