Comparison of 2D/3D Features and Their Adaptive Score Level Fusion for 3D Face Recognition

3D face has been introduced in the literature to deal with the unsolved issues of 2D face recognition, namely lighting and pose variations. In this paper, we study and compare the distinctiveness of features extracted from both the registered 2D face images and 3D face models. Sparse Representation Classifier (SRC) is exploited to calculate all similarity measures which are compared with the ones by a baseline of Nearest Neighbor (NN). As individual features of 2D and 3D are far from distinctive for discriminating human faces, we further present an adaptive score level fusion strategy for multimodal 2D-3D face recognition. The novel fusion strategy consists of an offline and an online weight learning process, both of which automatically select the most relevant weights of all the scores for each probe face in each modality. The weights calculated offline are based on the EER value of each type of features, while the online ones are dynamically obtained according to matching scores. Both types of weights are then fused to generate a final weight. Tested on the complete FRGC v2.0 dataset, the best rank-one recognition rate using only 3D or 2D features is 79.72% and 77.89%, respectively; while the new proposed adaptive fusion strategy achieves 95.48% with a 97.03% verification rate at 0.001 FAR, highlighting the benefit of exploring both 3D and 2D clues as well as the effectiveness of our adaptive fusion strategy.

[1]  Chin-Seng Chua,et al.  Facial feature detection and face recognition from 2D and 3D images , 2002, Pattern Recognit. Lett..

[2]  Remco C. Veltkamp,et al.  A Survey of 3D Face Recognition Methods , 2005, AVBPA.

[3]  S Marcelja,et al.  Mathematical description of the responses of simple cortical cells. , 1980, Journal of the Optical Society of America.

[4]  Daniel Rueckert,et al.  Evaluation of automatic 4D face recognition using surface and texture registration , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[5]  Patrick J. Flynn,et al.  Overview of the face recognition grand challenge , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[6]  Bülent Sankur,et al.  Representation Plurality and Fusion for 3-D Face Recognition , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[7]  Ke Huang,et al.  Sparse Representation for Signal Classification , 2006, NIPS.

[8]  Michael G. Strintzis,et al.  Use of depth and colour eigenfaces for face recognition , 2003, Pattern Recognit. Lett..

[9]  Ioannis A. Kakadiaris,et al.  Three-Dimensional Face Recognition in the Presence of Facial Expressions: An Annotated Deformable Model Approach , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Matti Pietikäinen,et al.  Face Recognition with Local Binary Patterns , 2004, ECCV.

[11]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Patrick J. Flynn,et al.  Using multi-instance enrollment to improve performance of 3D face recognition , 2008, Comput. Vis. Image Underst..

[13]  Anil K. Jain,et al.  Integrating Range and Texture Information for 3D Face Recognition , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[14]  Liming Chen,et al.  New Experiments on ICP-Based 3D Face Recognition and Authentication , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[15]  Patrick J. Flynn,et al.  A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition , 2006, Comput. Vis. Image Underst..

[16]  Afzal Godil,et al.  Face recognition using 3D facial shape and color map information: comparison and combination , 2004, SPIE Defense + Commercial Sensing.

[17]  Marina L. Gavrilova,et al.  FES: A System for Combining Face, Ear and Signature Biometrics Using Rank Level Fusion , 2008, Fifth International Conference on Information Technology: New Generations (itng 2008).

[18]  Mohammed Bennamoun,et al.  Keypoint Detection and Local Feature Matching for Textured 3D Face Recognition , 2007, International Journal of Computer Vision.

[19]  Anil K. Jain,et al.  Matching 2.5D face scans to 3D models , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Liming Chen,et al.  A coarse-to-fine curvature analysis-based rotation invariant 3D face landmarking , 2009, 2009 IEEE 3rd International Conference on Biometrics: Theory, Applications, and Systems.

[21]  Gérard G. Medioni,et al.  Performance of Geometrix ActiveID^TM 3D Face Recognition Engine on the FRGC Data , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[22]  Berk Gökberk,et al.  Comparative Analysis of Decision-level Fusion Algorithms for 3D Face Recognition , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[23]  Norbert Krüger,et al.  Face recognition by elastic bunch graph matching , 1997, Proceedings of International Conference on Image Processing.

[24]  Jiuchao Feng,et al.  KFCE: A dictionary generation algorithm for sparse representation , 2009, Signal Process..

[25]  Anil K. Jain,et al.  Large-scale evaluation of multimodal biometric authentication using state-of-the-art systems , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Berk Gökberk,et al.  Rank-based decision fusion for 3D shape-based face recognition , 2005, SIU 2005.

[27]  Patrick J. Flynn,et al.  3D Face Recognition with Region Committee Voting , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[28]  Christoph von der Malsburg,et al.  Strategies and Benefits of Fusion of 2D and 3D Face Recognition , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[29]  Mohammed Bennamoun,et al.  A survey of approaches to three-dimensional face recognition , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[30]  Shihong Lao,et al.  3D template matching for pose invariant face recognition using 3D facial model built with isoluminance line based stereo vision , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[31]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[32]  S. Zucker,et al.  Differential Geometry from the Frenet Point of View: Boundary Detection, Stereo, Texture and Color , 2006, Handbook of Mathematical Models in Computer Vision.

[33]  Marc Acheroy,et al.  Face verification from 3D and grey level clues , 2001, Pattern Recognit. Lett..

[34]  Arun Ross,et al.  An introduction to biometric recognition , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[35]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[36]  Raffaella Lanzarotti,et al.  Face Recognition Based on 2D and 3D Features , 2007, KES.

[37]  Mohammed Bennamoun,et al.  An Efficient Multimodal 2D-3D Hybrid Approach to Automatic Face Recognition , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.