Matching Depth to RGB for Boosting Face Verification

Low cost RGB-D sensors like Kinect and RealSense enable easy acquisition of both RGB (i.e., texture) and depth images of human faces. Many methods have been proposed to improve the RGB-to-RGB face matcher by fusing it with the Depth-to-Depth face matcher. Yet, few efforts have been devoted to the matching between RGB and Depth face images. In this paper, we propose two deep convolutional neural network (DCNN) based approaches to Depth-to-RGB face recognition, and compare their performance in terms of face verification accuracy. We further combine the Depth-to-RGB matcher with the RGB-to-RGB matcher via score-level fusion. Evaluation experiments on two databases demonstrate that matching depth to RGB does boost face verification accuracy.

[1]  Yu Qiao,et al.  Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks , 2016, IEEE Signal Processing Letters.

[2]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Xiaogang Wang,et al.  Deep Learning Face Representation from Predicting 10,000 Classes , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Mike Y. Chen,et al.  RealSense: directional interaction for proximate mobile sharing using built-in orientation sensors , 2013, MM '13.

[6]  Li Bai,et al.  Cosine Similarity Metric Learning for Face Verification , 2010, ACCV.

[7]  Shang-Hong Lai,et al.  Accurate and robust face recognition from RGB-D images with a deep learning approach , 2016, BMVC.

[8]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[9]  Isabelle Bloch.  Information combination operators for data fusion: a comparative review with classification , 1996, IEEE Trans. Syst. Man Cybern. Part A.

[10]  Richa Singh,et al.  RGB-D Face Recognition With Texture and Attribute Features , 2014, IEEE Transactions on Information Forensics and Security.

[11]  Xiaogang Wang,et al.  Deeply learned face representations are sparse, selective, and robust , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Mohammed Bennamoun,et al.  An RGB-D based image set classification for robust face recognition from Kinect data , 2016, Neurocomputing.

[13]  Yu Qiao,et al.  A Discriminative Feature Learning Approach for Deep Face Recognition , 2016, ECCV.

[14]  Yann LeCun,et al.  Learning a similarity metric discriminatively, with application to face verification , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[15]  Xiaogang Wang,et al.  Deep Learning Face Representation by Joint Identification-Verification , 2014, NIPS.

[16]  Zhigang Zhu,et al.  Face Recognition by 3D Registration for the Visually Impaired Using a RGB-D Sensor , 2014, ECCV Workshops.

[17]  Yunhong Wang,et al.  Lock3DFace: A large-scale database of low-cost Kinect 3D faces , 2016, 2016 International Conference on Biometrics (ICB).

[18]  Guodong Guo,et al.  Face recognition robust to head pose changes based on the RGB-D sensor , 2013, 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS).