Still to video face recognition using a heterogeneous matching approach

In this paper, we address the problem of still-to-video (S2V) face recognition. Still images usually have high qualities, captured from cooperative users under controlled environment, such as the mugshot photos. On the contrary, video clips may be acquired with low resolutions and low qualities, from non-cooperative users under uncontrolled environment. Because of these significant differences, we consider the S2V as a heterogeneous matching problem, and propose to develop a method to bridge the gap between the two heterogeneous modalities. A Grassmann manifold learning method is developed to construct subspaces for the purpose of bridging the gap between the two face modalities smoothly. We conduct extensive experiments on two large scale benchmark databases, COX-S2V and PaSC, with different recognition tasks: face identification and verification. The experimental results show that the proposed approach outperforms the state-of-the-art methods under the same experimental settings.

[1]  Masashi Sugiyama,et al.  Dimensionality Reduction of Multimodal Labeled Data by Local Fisher Discriminant Analysis , 2007, J. Mach. Learn. Res..

[2]  Changsong Liu,et al.  Still-to-video face recognition in unconstrained environments , 2015, Electronic Imaging.

[3]  Yuan Shi,et al.  Geodesic flow kernel for unsupervised domain adaptation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Chunheng Wang,et al.  Still-to-Video face recognition via weighted scenario oriented discriminant analysis , 2014, IEEE International Joint Conference on Biometrics.

[5]  Shiguang Shan,et al.  Joint sparse representation for video-based face recognition , 2014, Neurocomputing.

[6]  Xiaogang Wang,et al.  Lighting and Pose Robust Face Sketch Synthesis , 2010, ECCV.

[7]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[8]  David Zhang,et al.  From Point to Set: Extend the Learning of Distance Metrics , 2013, 2013 IEEE International Conference on Computer Vision.

[9]  Likun Huang,et al.  Face recognition based on image sets , 2014 .

[10]  Bruce A. Draper,et al.  The challenge of face recognition from digital point-and-shoot cameras , 2013, 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[11]  Geoffrey E. Hinton,et al.  Neighbourhood Components Analysis , 2004, NIPS.

[12]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[13]  Shiguang Shan,et al.  Learning Euclidean-to-Riemannian Metric for Point-to-Set Classification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  K.A. Gallivan,et al.  Efficient algorithms for inferences on Grassmann manifolds , 2004, IEEE Workshop on Statistical Signal Processing, 2003.

[15]  Larry S. Davis,et al.  Covariance discriminative learning: A natural and efficient approach to image set classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[17]  Anil K. Jain,et al.  Heterogeneous Face Recognition: Matching NIR to Visible Light Images , 2010, 2010 20th International Conference on Pattern Recognition.

[18]  Abdul Kadir,et al.  Special Issue: Recent Advances in Plant Leaf Classification , 2015, CVPR 2015.

[19]  Shiguang Shan,et al.  Benchmarking Still-to-Video Face Recognition via Partial and Local Linear Discriminant Analysis on COX-S2V Dataset , 2012, ACCV.

[20]  Anil K. Jain,et al.  Heterogeneous Face Recognition Using Kernel Prototype Similarities , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Tsuhan Chen,et al.  Video-based face recognition using adaptive hidden Markov models , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[22]  Dahua Lin,et al.  Inter-modality Face Recognition , 2006, ECCV.

[23]  Shengcai Liao,et al.  Coupled Discriminant Analysis for Heterogeneous Face Recognition , 2012, IEEE Transactions on Information Forensics and Security.

[24]  Shiguang Shan,et al.  Face video retrieval with image query via hashing across Euclidean space and Riemannian manifold , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Alan Edelman,et al.  The Geometry of Algorithms with Orthogonality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[26]  Josef Kittler,et al.  Discriminative Learning and Recognition of Image Set Classes Using Canonical Correlations , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Xiaogang Wang,et al.  Coupled information-theoretic encoding for face photo-sketch recognition , 2011, CVPR 2011.

[28]  Chunheng Wang,et al.  Scenario oriented discriminant analysis for still-to-video face recognition , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[29]  Stan Z. Li,et al.  Coupled Spectral Regression for matching heterogeneous faces , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Inderjit S. Dhillon,et al.  Information-theoretic metric learning , 2006, ICML '07.