Boosting 3D LBP-Based Face Recognition by Fusing Shape and Texture Descriptors on the Mesh

In this paper, we present a novel approach for fusing shape and texture local binary patterns (LBPs) on a mesh for 3D face recognition. Using a recently proposed framework, we compute LBP directly on the face mesh surface, then we construct a grid of the regions on the facial surface that can accommodate global and partial descriptions. Compared with its depth-image counterpart, our approach is distinguished by the following features: 1) inherits the intrinsic advantages of mesh surface (e.g., preservation of the full geometry); 2) does not require normalization; and 3) can accommodate partial matching. In addition, it allows early level fusion of texture and shape modalities. Through experiments conducted on the BU-3DFE and Bosphorus databases, we assess different variants of our approach with regard to facial expressions and missing data, also in comparison to the state-of-the-art solutions.

[1]  Radu Horaud,et al.  Surface feature detection and description with applications to mesh matching , 2009, CVPR.

[2]  Alberto Del Bimbo,et al.  The Mesh-LBP: Computing Local Binary Patterns on Discrete Manifolds , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[3]  Matti Pietikäinen,et al.  CS-3DLBP and geometry based person independent 3D facial action unit detection , 2013, 2013 International Conference on Biometrics (ICB).

[4]  Ashraf A. Kassim,et al.  On the simultaneous recognition of identity and expression from BU-3DFE datasets , 2012, Pattern Recognit. Lett..

[5]  L. Akarun,et al.  A 3D Face Recognition System for Expression and Occlusion Invariance , 2008, 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems.

[6]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Tieniu Tan,et al.  Combining Statistics of Geometrical and Correlative Features for 3D Face Recognition , 2006, BMVC.

[8]  Matti Pietikäinen,et al.  Face Recognition with Local Binary Patterns , 2004, ECCV.

[9]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Yanfeng Sun,et al.  3D face recognition using local binary patterns , 2013, Signal Process..

[11]  Hassen Drira,et al.  3D Face Recognition under Expressions, Occlusions, and Pose Variations , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Patrick J. Flynn,et al.  An evaluation of multimodal 2D+3D face biometrics , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Christoph von der Malsburg,et al.  Strategies and Benefits of Fusion of 2D and 3D Face Recognition , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[14]  Stefanos Zafeiriou,et al.  Local normal binary patterns for 3D facial action unit detection , 2012, 2012 19th IEEE International Conference on Image Processing.

[15]  Stan Z. Li,et al.  Learning to Fuse 3D+2D Based Face Recognition at Both Feature and Decision Levels , 2005, AMFG.

[16]  Stefanos Zafeiriou,et al.  Binary Pattern Analysis for 3D Facial Action Unit Detection , 2012, BMVC.

[17]  Alberto Del Bimbo,et al.  Matching 3D face scans using interest points and local histogram descriptors , 2013, Comput. Graph..

[18]  Marc Acheroy,et al.  Face verification from 3D and grey level clues , 2001, Pattern Recognit. Lett..

[19]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[20]  Arun Ross,et al.  An introduction to biometric recognition , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[21]  Paul Suetens,et al.  meshSIFT: Local surface features for 3D face recognition under expression variations and partial data , 2013, Comput. Vis. Image Underst..

[22]  Vincent Lepetit,et al.  A fast local descriptor for dense matching , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Di Huang,et al.  3-D Face Recognition Using eLBP-Based Facial Description and Local Feature Hybrid Matching , 2012, IEEE Transactions on Information Forensics and Security.

[24]  Jun Wang,et al.  A 3D facial expression database for facial behavior research , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[25]  Di Huang,et al.  Comparison of 2D/3D Features and Their Adaptive Score Level Fusion for 3D Face Recognition , 2010 .

[26]  Gérard G. Medioni,et al.  Performance of Geometrix ActiveID^TM 3D Face Recognition Engine on the FRGC Data , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[27]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[28]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[29]  Liming Chen,et al.  3D facial expression recognition via multiple kernel learning of Multi-Scale Local Normal Patterns , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[30]  Mohammed Bennamoun,et al.  An Efficient Multimodal 2D-3D Hybrid Approach to Automatic Face Recognition , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Patrick J. Flynn,et al.  Multi-Modal 2D and 3D Biometrics for Face Recognition , 2003, AMFG.

[32]  Bülent Sankur,et al.  Representation Plurality and Fusion for 3-D Face Recognition , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[33]  Mohammed Bennamoun,et al.  An efficient 3D face recognition approach based on the fusion of novel local low-level features , 2013, Pattern Recognit..

[34]  Olgica Milenkovic,et al.  Subspace Pursuit for Compressive Sensing Signal Reconstruction , 2008, IEEE Transactions on Information Theory.

[35]  Faisal R. Al-Osaimi,et al.  Spatially Optimized Data-Level Fusion of Texture and Shape for Face Recognition , 2012, IEEE Transactions on Image Processing.

[36]  Paul Suetens,et al.  Feature detection on 3D face surfaces for pose normalisation and recognition , 2010, 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[37]  Arun Ross,et al.  Information fusion in biometrics , 2003, Pattern Recognit. Lett..

[38]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[39]  Alberto Del Bimbo,et al.  The Mesh-LBP: A Framework for Extracting Local Binary Patterns From Discrete Manifolds , 2015, IEEE Transactions on Image Processing.

[40]  Alberto Del Bimbo,et al.  3D Face Recognition Using Isogeodesic Stripes , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Alberto Del Bimbo,et al.  Representing 3D texture on mesh manifolds for retrieval and recognition applications , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Mohammed Bennamoun,et al.  Keypoint Detection and Local Feature Matching for Textured 3D Face Recognition , 2007, International Journal of Computer Vision.

[43]  Liming Chen,et al.  Expression robust 3D face recognition via mesh-based histograms of multiple order surface differential quantities , 2011, 2011 18th IEEE International Conference on Image Processing.

[44]  Patrick J. Flynn,et al.  A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition , 2006, Comput. Vis. Image Underst..

[45]  Liming Chen,et al.  Towards 3D Face Recognition in the Real: A Registration-Free Approach Using Fine-Grained Matching of 3D Keypoint Descriptors , 2014, International Journal of Computer Vision.

[46]  Anil K. Jain,et al.  Deformation Modeling for Robust 3D Face Matching , 2006, CVPR.

[47]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[48]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  Di Huang,et al.  Local Binary Patterns and Its Application to Facial Image Analysis: A Survey , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).