Pose-robust representation for face verification in unconstrained videos

Face recognition in unconstrained videos has been actively studied with the increasing popularity of surveillance and personal cameras. Compared with the traditional task of face recognition in images, recognizing faces in unconstrained videos is much more challenging due to the large variations in poses, expressions and lighting conditions. To handle varying poses in the videos, in this paper, we propose a two-level representation approach for face verification in unconstrained videos. Specifically, we first recover the full-pose face representation for each video which contains all the pose categories ranging from the most left to the most right profile faces. The missing poses are synthesized using keyframes of known ones. Then, we further propose a cross-pose video pair representation for face verification task, which consists of the similarity scores of all frame-level pairs across two videos. Extensive experiments on the benchmark YouTube Faces (YTF) dataset clearly demonstrate the effectiveness of our proposed method.

[1]  Tsuhan Chen,et al.  Pose-robust face recognition using geometry assisted probabilistic modeling , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[2]  Arun Ross,et al.  Face Recognition in Video: Adaptive Fusion of Multiple Matchers , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Trevor Darrell,et al.  Face Recognition from Long-Term Observations , 2002, ECCV.

[4]  Rama Chellappa,et al.  Probabilistic recognition of human faces from video , 2002, Proceedings. International Conference on Image Processing.

[5]  Yongsheng Gao,et al.  Face recognition across pose: A review , 2009, Pattern Recognit..

[6]  Simon Lucey,et al.  Face alignment through subspace constrained mean-shifts , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[7]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Gang Hua,et al.  Probabilistic Elastic Matching for Pose Variant Face Verification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Dong Xu,et al.  Patch Distribution Compatible Semisupervised Dimension Reduction for Face and Human Gait Recognition , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[11]  Shiguang Shan,et al.  Image sets alignment for Video-Based Face Recognition , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Tal Hassner,et al.  Face recognition in unconstrained videos with matched background similarity , 2011, CVPR 2011.

[13]  Jian Sun,et al.  An associate-predict model for face recognition , 2011, CVPR 2011.

[14]  Yaniv Taigman,et al.  Descriptor Based Methods in the Wild , 2008 .

[15]  Lior Wolf,et al.  The SVM-Minus Similarity Score for Video Face Recognition , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Tal Hassner,et al.  Multiple One-Shots for Utilizing Class Label Information , 2009, BMVC.

[17]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Yang Wang,et al.  Enforcing convexity for improved alignment with constrained local models , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[20]  Rama Chellappa,et al.  Face Tracking and Recognition in Video , 2011, Handbook of Face Recognition.

[21]  Chengjun Liu,et al.  Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition , 2002, IEEE Trans. Image Process..

[22]  Shiguang Shan,et al.  Fusing Robust Face Region Descriptors via Multiple Metric Learning for Face Recognition in the Wild , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Dong Xu,et al.  Face and Human Gait Recognition Using Image-to-Class Distance , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[24]  Wen Gao,et al.  Manifold-Manifold Distance with application to face recognition based on image set , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[26]  Shree K. Nayar,et al.  Attribute and simile classifiers for face verification , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[27]  Dong Xu,et al.  Multilinear Discriminant Analysis for Face Recognition , 2007, IEEE Transactions on Image Processing.

[28]  Tsuhan Chen,et al.  Video-based face recognition using adaptive hidden Markov models , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[29]  Ken-ichi Maeda,et al.  Face recognition using temporal image sequence , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[30]  Shiguang Shan,et al.  Side-Information based Linear Discriminant Analysis for Face Recognition , 2011, BMVC.

[31]  Peter N. Belhumeur,et al.  Tom-vs-Pete Classifiers and Identity-Preserving Alignment for Face Verification , 2012, BMVC.