Representing 3D texture on mesh manifolds for retrieval and recognition applications

In this paper, we present and experiment a novel approach for representing texture of 3D mesh manifolds using local binary patterns (LBP). Using a recently proposed framework [37], we compute LBP directly on the mesh surface, either using geometric or photometric appearance. Compared to its depth-image counterpart, our approach is distinguished by the following features: a) inherits the intrinsic advantages of mesh surface (e.g., preservation of the full geometry); b) does not require normalization; c) can accommodate partial matching. In addition, it allows early-level fusion of the geometry and photometric texture modalities. Through experiments conducted on two application scenarios, namely, 3D texture retrieval and 3D face recognition, we assess the effectiveness of the proposed solution with respect to state of the art approaches.

[1]  Iasonas Kokkinos,et al.  Intrinsic shape context descriptors for deformable shapes , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Patrick J. Flynn,et al.  An evaluation of multimodal 2D+3D face biometrics , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Andrew E. Johnson,et al.  Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Ammad Ali,et al.  Face Recognition with Local Binary Patterns , 2012 .

[5]  Bernard Chazelle,et al.  Shape distributions , 2002, TOGS.

[6]  Mohammed Bennamoun,et al.  Keypoint Detection and Local Feature Matching for Textured 3D Face Recognition , 2007, International Journal of Computer Vision.

[7]  Yuming Zhao,et al.  Fast Tracking of Object Contour Based on Color and Texture , 2009, Int. J. Pattern Recognit. Artif. Intell..

[8]  Patrick J. Flynn,et al.  Multi-Modal 2D and 3D Biometrics for Face Recognition , 2003, AMFG.

[9]  Jiebo Luo,et al.  Heterogeneous feature machines for visual recognition , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[10]  Mohammed Bennamoun,et al.  An efficient 3D face recognition approach based on the fusion of novel local low-level features , 2013, Pattern Recognit..

[11]  Peijun Li,et al.  Rotation Invariant Texture Measured by Local Binary Pattern for Remote Sensing Image Classification , 2010, 2010 Second International Workshop on Education Technology and Computer Science.

[12]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[13]  Liming Chen,et al.  3D facial expression recognition via multiple kernel learning of Multi-Scale Local Normal Patterns , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[14]  Jun Wang,et al.  A 3D facial expression database for facial behavior research , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[15]  Matti Pietikäinen,et al.  CS-3DLBP and geometry based person independent 3D facial action unit detection , 2013, 2013 International Conference on Biometrics (ICB).

[16]  Ashraf A. Kassim,et al.  On the simultaneous recognition of identity and expression from BU-3DFE datasets , 2012, Pattern Recognit. Lett..

[17]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Liming Chen,et al.  Expression robust 3D face recognition via mesh-based histograms of multiple order surface differential quantities , 2011, 2011 18th IEEE International Conference on Image Processing.

[20]  Patrick J. Flynn,et al.  A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition , 2006, Comput. Vis. Image Underst..

[21]  Alfred Stein,et al.  Multivariate texture‐based segmentation of remotely sensed imagery for extraction of objects and their uncertainty , 2005 .

[22]  Mohammed Bennamoun,et al.  An Efficient Multimodal 2D-3D Hybrid Approach to Automatic Face Recognition , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Faisal R. Al-Osaimi,et al.  Spatially Optimized Data-Level Fusion of Texture and Shape for Face Recognition , 2012, IEEE Transactions on Image Processing.

[24]  Alberto Del Bimbo,et al.  The Mesh-LBP: A Framework for Extracting Local Binary Patterns From Discrete Manifolds , 2015, IEEE Transactions on Image Processing.

[25]  L. Akarun,et al.  A 3D Face Recognition System for Expression and Occlusion Invariance , 2008, 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems.

[26]  Gérard G. Medioni,et al.  Performance of Geometrix ActiveID^TM 3D Face Recognition Engine on the FRGC Data , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[27]  Majid Mirmehdi,et al.  Archive Film Restoration Based on Spatiotemporal Random Walks , 2010, ECCV.

[28]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Xihong Wu,et al.  Boosting Local Binary Pattern (LBP)-Based Face Recognition , 2004, SINOBIOMETRICS.

[30]  Matti Pietikäinen,et al.  Optimising Colour and Texture Features for Real-time Visual Inspection , 2002, Pattern Analysis & Applications.

[31]  Stefanos Zafeiriou,et al.  Local normal binary patterns for 3D facial action unit detection , 2012, 2012 19th IEEE International Conference on Image Processing.

[32]  Yaniv Taigman,et al.  Descriptor Based Methods in the Wild , 2008 .

[33]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[34]  Arun Ross,et al.  Information fusion in biometrics , 2003, Pattern Recognit. Lett..

[35]  Arun Ross,et al.  An introduction to biometric recognition , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[36]  Vijayalakshmi Atluri,et al.  Texture-Based Remote-Sensing Image Segmentation , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[37]  Stan Z. Li,et al.  Learning to Fuse 3D+2D Based Face Recognition at Both Feature and Decision Levels , 2005, AMFG.

[38]  Stefanos Zafeiriou,et al.  Binary Pattern Analysis for 3D Facial Action Unit Detection , 2012, BMVC.

[39]  Alberto Del Bimbo,et al.  Matching 3D face scans using interest points and local histogram descriptors , 2013, Comput. Graph..