Local feature hierarchy for face recognition across pose and illumination

Even though face recognition in frontal view and normal lighting condition works very well, the performance degenerates sharply in extreme conditions. In real applications, both the lighting and pose variation will always be encountered at the same time. Accordingly we propose an end-to-end face recognition method to deal with pose and illumination simultaneously based on convolutional neural networks where the discriminative nonlinear features that are invariant to pose and illumination are extracted. Normally the global structure for images taken in different views is quite diverse. Therefore we propose to use the 1 × 1 convolutional kernel to extract the local features. Furthermore the parallel multi-stream multi-layer 1 × 1 convolution network is developed to extract multi-hierarchy features. In the experiments we obtained the average face recognition rate of 96.9% on multiPIE dataset, which improves the state-of-the-art of face recognition across poses and illumination by 7.5%. Especially for profile-wise positions, the average recognition rate of our proposed network is 97.8%, which increases the state-of-the-art recognition rate by 19%.

[1]  Xiaogang Wang,et al.  Deep Learning Identity-Preserving Face Space , 2013, 2013 IEEE International Conference on Computer Vision.

[2]  Xiaoyue Jiang,et al.  Learning from Real Images to Model Lighting Variations for Face Images , 2008, ECCV.

[3]  Shiguang Shan,et al.  Multi-View Discriminant Analysis , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Anil K. Jain,et al.  Handbook of Face Recognition, 2nd Edition , 2011 .

[5]  J. Shawe-Taylor,et al.  Multi-View Canonical Correlation Analysis , 2010 .

[6]  Takeo Kanade,et al.  Multi-PIE , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[7]  Xiaoyang Tan,et al.  Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions , 2007, IEEE Transactions on Image Processing.

[8]  Journal of the Optical Society of America , 1950, Nature.

[9]  Jonghyun Choi,et al.  Multi-Directional Multi-Level Dual-Cross Patterns for Robust Face Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Xiaogang Wang,et al.  Multi-View Perceptron: a Deep Model for Learning Face Identity and View Representations , 2014, NIPS.

[11]  Dacheng Tao,et al.  Multi-Task Pose-Invariant Face Recognition , 2015, IEEE Transactions on Image Processing.

[12]  David W. Jacobs,et al.  Generalized Multiview Analysis: A discriminative latent space , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Sébastien Marcel,et al.  EVALUATION IN MOBILE ENVIRONMENT , 2013 .

[15]  Dorin Comaniciu,et al.  Total variation models for variable lighting face recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Shuicheng Yan,et al.  An HOG-LBP human detector with partial occlusion handling , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[17]  Wen Gao,et al.  Coupled Bias–Variance Tradeoff for Cross-Pose Face Recognition , 2012, IEEE Transactions on Image Processing.

[18]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Yajie Tian,et al.  Handbook of face recognition , 2003 .

[20]  Shiguang Shan,et al.  Multi-view Deep Network for Cross-View Classification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  D. Jacobs,et al.  Bypassing synthesis: PLS for face recognition with pose, low-resolution and sketch , 2011, CVPR 2011.

[23]  Jonathan Warrell,et al.  Tied Factor Analysis for Face Recognition across Large Pose Differences , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  P. Hanrahan,et al.  On the relationship between radiance and irradiance: determining the illumination from images of a convex Lambertian object. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[25]  Hossein Mobahi,et al.  Toward a Practical Face Recognition System: Robust Alignment and Illumination by Sparse Representation , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Carlos D. Castillo,et al.  Wide-baseline stereo for face recognition with large pose variation , 2011, CVPR 2011.

[27]  Sébastien Marcel,et al.  The 2013 speaker recognition evaluation in mobile environment , 2013, 2013 International Conference on Biometrics (ICB).

[28]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[29]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.