Enhance deep learning performance in face recognition

Deep convolutional neural networks (CNNs) based face recognition approaches have been dominating the field. The success of CNNs is attributed to their ability to learn rich image representations. But training CNNs relies on estimating millions of parameters and requires a very large number of annotated training images. A widely-used alternative is to fine-tune the CNN that has been pre-trained using a large set of labeled images. However, we show that fine-tuning pre-trained CNNs cannot provide satisfactory face recognition performance when training and testing datasets have large differences. To address this problem, we propose to improve the face recognition performance of CNNs by using non-CNN features. Extensive experiments are conducted on LFW and FRGC databases using the pre-trained CNN model, VGG-Face. Results show that the complementary information contained in non-CNN features greatly improves the face verification rate/accuracy of CNNs on LFW and FRGC databases. Furthermore, we show that non-CNN features are more effective in enhancing the performance of pre-trained CNNs than fine-tuning.

[1]  Xudong Jiang,et al.  Classwise Sparse and Collaborative Patch Representation for Face Recognition , 2016, IEEE Trans. Image Process..

[2]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[3]  Patrick J. Flynn,et al.  Overview of the face recognition grand challenge , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[4]  Bruce A. Draper,et al.  A meta-analysis of face recognition covariates , 2009, 2009 IEEE 3rd International Conference on Biometrics: Theory, Applications, and Systems.

[5]  Zhaoqiang Xia,et al.  Towards Facial Expression Recognition in the Wild: A New Database and Deep Recognition System , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[6]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[7]  Chengjun Liu,et al.  Comparative assessment of content-based face image retrieval in different color spaces , 2005, Int. J. Pattern Recognit. Artif. Intell..

[8]  Alexei A. Efros,et al.  Unbiased look at dataset bias , 2011, CVPR 2011.

[9]  Xudong Jiang,et al.  An effective color space for face recognition , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[10]  Chengjun Liu,et al.  Comparative assessment of content-based face image retrieval in different color spaces , 2005, Int. J. Pattern Recognit. Artif. Intell..

[11]  Andrew Zisserman,et al.  Deep Face Recognition , 2015, BMVC.

[12]  Xudong Jiang,et al.  Constructing and training feed-forward neural networks for pattern classification , 2003, Pattern Recognit..

[13]  Xudong Jiang,et al.  A Color Channel Fusion Approach for Face Recognition , 2015, IEEE Signal Processing Letters.

[14]  Li-Jia Li,et al.  Multi-view Face Detection Using Deep Convolutional Neural Networks , 2015, ICMR.

[15]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[16]  Yong Man Ro,et al.  Color Face Recognition for Degraded Face Images , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[17]  Xiaogang Wang,et al.  Deep Learning Face Representation from Predicting 10,000 Classes , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Xudong Jiang.  On orientation and anisotropy estimation for online fingerprint authentication , 2005, IEEE Transactions on Signal Processing.

[19]  Xudong Jiang,et al.  Sparse and Dense Hybrid Representation via Dictionary Decomposition for Face Recognition , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Yu Qiao,et al.  Gender and Smile Classification Using Deep Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[21]  Yoshua Bengio,et al.  Deep Learning of Representations for Unsupervised and Transfer Learning , 2011, ICML Unsupervised and Transfer Learning.

[22]  Xudong Jiang,et al.  Extracting image orientation feature by using integration operator , 2007, Pattern Recognit..

[23]  Ivan Laptev,et al.  Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Xudong Jiang,et al.  Linear Subspace Learning-Based Dimensionality Reduction , 2011, IEEE Signal Processing Magazine.

[25]  Pascal Vincent,et al.  The Difficulty of Training Deep Architectures and the Effect of Unsupervised Pre-Training , 2009, AISTATS.

[26]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Xudong Jiang,et al.  Asymmetric Principal Component and Discriminant Analyses for Pattern Classification , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.