Boosting 3D LBP-based face recognition by fusing shape and texture descriptors on the mesh

In this paper, we present a novel approach for fusing shape and texture local binary patterns (LBP) for 3D face recognition. Using the framework proposed in [1], we compute LBP directly on the face mesh surface, then we construct a grid of the regions on the facial surface that can accommodate global and partial descriptions. Compared to its depth-image counterpart, our approach is distinguished by the following features: a) inherits the intrinsic advantages of mesh surface; b) does not require normalization; c) can accommodate partial matching. In addition, it allows early-level fusion of texture and shape modalities. Through experiments conducted on the BU-3DFE and Bosphorus databases, we assess different variants of our approach with regard to facial expressions and missing data.

[1]  L. Akarun,et al.  A 3D Face Recognition System for Expression and Occlusion Invariance , 2008, 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems.

[2]  Bülent Sankur,et al.  Representation Plurality and Fusion for 3-D Face Recognition , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[3]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[4]  Paul Suetens,et al.  meshSIFT: Local surface features for 3D face recognition under expression variations and partial data , 2013, Comput. Vis. Image Underst..

[5]  Mohammed Bennamoun,et al.  An efficient 3D face recognition approach based on the fusion of novel local low-level features , 2013, Pattern Recognit..

[6]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Marc Acheroy,et al.  Face verification from 3D and grey level clues , 2001, Pattern Recognit. Lett..

[8]  Arun Ross,et al.  An introduction to biometric recognition , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[9]  Paul Suetens,et al.  Feature detection on 3D face surfaces for pose normalisation and recognition , 2010, 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[10]  Stefanos Zafeiriou,et al.  Local normal binary patterns for 3D facial action unit detection , 2012, 2012 19th IEEE International Conference on Image Processing.

[11]  Alberto Del Bimbo,et al.  The Mesh-LBP: Computing Local Binary Patterns on Discrete Manifolds , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[12]  Faisal R. Al-Osaimi,et al.  Spatially Optimized Data-Level Fusion of Texture and Shape for Face Recognition , 2012, IEEE Transactions on Image Processing.

[13]  Liming Chen,et al.  Expression robust 3D face recognition via mesh-based histograms of multiple order surface differential quantities , 2011, 2011 18th IEEE International Conference on Image Processing.

[14]  Patrick J. Flynn,et al.  A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition , 2006, Comput. Vis. Image Underst..

[15]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Alberto Del Bimbo,et al.  Representing 3D texture on mesh manifolds for retrieval and recognition applications , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Jun Wang,et al.  A 3D facial expression database for facial behavior research , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[18]  Di Huang,et al.  Comparison of 2D/3D Features and Their Adaptive Score Level Fusion for 3D Face Recognition , 2010 .

[19]  Mohammed Bennamoun,et al.  Keypoint Detection and Local Feature Matching for Textured 3D Face Recognition , 2007, International Journal of Computer Vision.

[20]  Hassen Drira,et al.  3D Face Recognition under Expressions, Occlusions, and Pose Variations , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Di Huang,et al.  Local Binary Patterns and Its Application to Facial Image Analysis: A Survey , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[23]  B. K. Julsing,et al.  Face Recognition with Local Binary Patterns , 2012 .

[24]  Tieniu Tan,et al.  Combining Statistics of Geometrical and Correlative Features for 3D Face Recognition , 2006, BMVC.

[25]  Stan Z. Li,et al.  Learning to Fuse 3D+2D Based Face Recognition at Both Feature and Decision Levels , 2005, AMFG.

[26]  Stefanos Zafeiriou,et al.  Binary Pattern Analysis for 3D Facial Action Unit Detection , 2012, BMVC.

[27]  Alberto Del Bimbo,et al.  Matching 3D face scans using interest points and local histogram descriptors , 2013, Comput. Graph..

[28]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Gérard G. Medioni,et al.  Performance of Geometrix ActiveID^TM 3D Face Recognition Engine on the FRGC Data , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[30]  Yanfeng Sun,et al.  3D face recognition using local binary patterns , 2013, Signal Process..

[31]  Patrick J. Flynn,et al.  An evaluation of multimodal 2D+3D face biometrics , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Christoph von der Malsburg,et al.  Strategies and Benefits of Fusion of 2D and 3D Face Recognition , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[33]  Vincent Lepetit,et al.  A fast local descriptor for dense matching , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Anil K. Jain,et al.  Deformation Modeling for Robust 3D Face Matching , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Di Huang,et al.  3-D Face Recognition Using eLBP-Based Facial Description and Local Feature Hybrid Matching , 2012, IEEE Transactions on Information Forensics and Security.

[36]  Alberto Del Bimbo,et al.  The Mesh-LBP: A Framework for Extracting Local Binary Patterns From Discrete Manifolds , 2015, IEEE Transactions on Image Processing.

[37]  Alberto Del Bimbo,et al.  3D Face Recognition Using Isogeodesic Stripes , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[39]  Liming Chen,et al.  Towards 3D Face Recognition in the Real: A Registration-Free Approach Using Fine-Grained Matching of 3D Keypoint Descriptors , 2014, International Journal of Computer Vision.

[40]  Patrick J. Flynn,et al.  Multi-Modal 2D and 3D Biometrics for Face Recognition , 2003, AMFG.

[41]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[42]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[43]  Liming Chen,et al.  3D facial expression recognition via multiple kernel learning of Multi-Scale Local Normal Patterns , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[44]  Arun Ross,et al.  Information fusion in biometrics , 2003, Pattern Recognit. Lett..

[45]  Mohammed Bennamoun,et al.  An Efficient Multimodal 2D-3D Hybrid Approach to Automatic Face Recognition , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Matti Pietikäinen,et al.  CS-3DLBP and geometry based person independent 3D facial action unit detection , 2013, 2013 International Conference on Biometrics (ICB).

[47]  Ashraf A. Kassim,et al.  On the simultaneous recognition of identity and expression from BU-3DFE datasets , 2012, Pattern Recognit. Lett..

[48]  R. Horaud,et al.  Surface feature detection and description with applications to mesh matching , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[49]  Olgica Milenkovic,et al.  Subspace Pursuit for Compressive Sensing Signal Reconstruction , 2008, IEEE Transactions on Information Theory.