Improving 2D Face Recognition via Discriminative Face Depth Estimation

As face recognition progresses from constrained scenarios to unconstrained scenarios, new challenges such as large pose, bad illumination, and partial occlusion, are encountered. While 3D or multi-modality RGB-D sensors are helpful for face recognition systems to achieve robustness against these challenges, the requirement of new sensors limits their application scenarios. In our paper, we propose a discriminative face depth estimation approach to improve 2D face recognition accuracies under unconstrained scenarios. Our discriminative depth estimation method uses a cascaded FCN and CNN architecture, in which FCN aims at recovering the depth from an RGB image, and CNN retains the separability of individual subjects. The estimated depth information is then used as a complementary modality to RGB for face recognition tasks. Experiments on two public datasets and a dataset we collect show that the proposed face recognition method using RGB and estimated depth information can achieve better accuracy than using RGB modality alone.

[1]  Kin-Man Lam,et al.  Depth Estimation of Face Images Using the Nonlinear Least-Squares Model , 2013, IEEE Transactions on Image Processing.

[2]  Qiang Wu,et al.  Real Time Complete Dense Depth Reconstruction for a Monocular Camera , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[3]  Chunhua Shen,et al.  Estimating Depth From Monocular Images as Classification Using Deep Fully Convolutional Residual Networks , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[4]  Samarth Bharadwaj,et al.  On RGB-D face recognition using Kinect , 2013, 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[5]  Richa Singh,et al.  RGB-D Face Recognition With Texture and Attribute Features , 2014, IEEE Transactions on Information Forensics and Security.

[6]  Weifeng Chen,et al.  Single-Image Depth Perception in the Wild , 2016, NIPS.

[7]  Yunhong Wang,et al.  Lock3DFace: A large-scale database of low-cost Kinect 3D faces , 2016, 2016 International Conference on Biometrics (ICB).

[8]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[9]  Anil K. Jain,et al.  Live face video vs. spoof face video: Use of moiré patterns to detect replay video attacks , 2015, 2015 International Conference on Biometrics (ICB).

[10]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[11]  Guosheng Lin,et al.  Deep convolutional neural fields for depth estimation from a single image , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Gao Wen,et al.  A Lighting Normalization Approach Exploiting Face Symmetry , 2013 .

[13]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[14]  Pratim Ghosh,et al.  Robust depth estimation for efficient 3D face reconstruction , 2008, ICIP 2008.

[15]  Yang Yang,et al.  Effective 3D face depth estimation from a single 2D face image , 2016, 2016 16th International Symposium on Communications and Information Technologies (ISCIT).

[16]  Nassir Navab,et al.  Deeper Depth Prediction with Fully Convolutional Residual Networks , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[17]  Rob Fergus,et al.  Depth Map Prediction from a Single Image using a Multi-Scale Deep Network , 2014, NIPS.

[18]  Kin-Man Lam,et al.  Depth Estimation of Face Images Based on the Constrained ICA Model , 2010, IEEE Transactions on Information Forensics and Security.

[19]  Xiaoming Liu,et al.  Demographic Estimation from Face Images: Human vs. Machine Performance , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Horst Bischof,et al.  Estimation of Face Depth Maps from Color Textures using Canonical Correlation Analysis , 2006 .

[21]  Ian D. Reid,et al.  Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Hu Tian,et al.  Depth estimation with convolutional conditional random field network , 2016, Neurocomputing.

[23]  Antonio Torralba,et al.  Depth Estimation from Image Structure , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Jean-Luc Dugelay,et al.  An Efficient LBP-Based Descriptor for Facial Depth Images Applied to Gender Recognition Using RGB-D Face Data , 2012, ACCV Workshops.

[25]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Singh Richa,et al.  RGB-D face recognition via learning-based reconstruction , 2016 .

[28]  Chunhua Shen,et al.  Depth and surface normal estimation from monocular images using regression on deep features and hierarchical CRFs , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[30]  Anil K. Jain,et al.  Cross-Database Face Antispoofing with Robust Feature Representation , 2016, CCBR.

[31]  Shiguang Shan,et al.  Heterogeneous Face Attribute Estimation: A Deep Multi-Task Learning Approach , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.