Multi-manifold metric learning for face recognition based on image sets

A multi-manifold metric learning (MMML) method is proposed.The distance metrics learned in the proposed MMML method are set-based metrics.The distance metrics learned in the proposed MMML method are person-specific.Experimental results show the effectiveness of the proposed MMML method. In this paper, we propose a new multi-manifold metric learning (MMML) method for the task of face recognition based on image sets. Different from most existing metric learning algorithms that learn the distance metric for measuring single images, our method aims to learn distance metrics to measure the similarity between manifold pairs. In our method, each image set is modeled as a manifold and then multiple distance metrics among different manifolds are learned. With these distance metrics, the intra-class manifold variations are minimized and inter-class manifold variations are maximized simultaneously. For each person, we learn a distance metric by using such a criterion that all the learned distance metrics are person-specific and thus more discriminative. Our method is extensively evaluated on three widely studied face databases, i.e., Honda/UCSD database, CMU MoBo database and YouTube Celebrities database, and compared to the state-of-the-arts. Experimental results are presented to show the effectiveness of the proposed method.

[1]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[2]  Gang Wang,et al.  Discriminative multi-manifold analysis for face recognition from a single training sample per person , 2011, 2011 International Conference on Computer Vision.

[3]  Stephen Lin,et al.  Marginal Fisher Analysis and Its Variants for Human Gait Recognition and Content- Based Image Retrieval , 2007, IEEE Transactions on Image Processing.

[4]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[5]  Qionghai Dai,et al.  Weighted Subspace Distance and Its Applications to Object Recognition and Retrieval With Image Sets , 2009, IEEE Signal Processing Letters.

[6]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[7]  Osamu Yamaguchi,et al.  Face Recognition Using Multi-viewpoint Patterns for Robot Vision , 2003, ISRR.

[8]  Liwei Wang,et al.  Subspace distance analysis with application to adaptive Bayesian algorithm for face recognition , 2006, Pattern Recognit..

[9]  Ming-Hsuan Yang,et al.  Incremental Learning for Robust Visual Tracking , 2008, International Journal of Computer Vision.

[10]  Rama Chellappa,et al.  Dictionary-Based Face Recognition from Video , 2012, ECCV.

[11]  Josef Kittler,et al.  Discriminative Learning and Recognition of Image Set Classes Using Canonical Correlations , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Michael I. Jordan,et al.  Distance Metric Learning with Application to Clustering with Side-Information , 2002, NIPS.

[13]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[14]  Ken-ichi Maeda,et al.  Face recognition using temporal image sequence , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[15]  Lior Wolf,et al.  Learning over Sets using Kernel Principal Angles , 2003, J. Mach. Learn. Res..

[16]  Trevor Darrell,et al.  Face recognition with image sets using manifold density divergence , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[17]  Daniel D. Lee,et al.  Grassmann discriminant analysis: a unifying view on subspace-based learning , 2008, ICML '08.

[18]  Ralph Gross,et al.  The CMU Motion of Body (MoBo) Database , 2001 .

[19]  David G. Stork,et al.  Pattern Classification , 1973 .

[20]  Trevor Darrell,et al.  Face Recognition from Long-Term Observations , 2002, ECCV.

[21]  Roberto Cipolla,et al.  An information-theoretic approach to face recognition from face motion manifolds , 2006, Image Vis. Comput..

[22]  Stephen Lin,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[24]  Mubarak Shah,et al.  Face Recognition in Movie Trailers via Mean Sequence Sparse Representation-Based Classification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Ruiping Wang,et al.  Manifold Discriminant Analysis , 2009, CVPR.

[26]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[28]  David J. Kriegman,et al.  Video-based face recognition using probabilistic appearance manifolds , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[29]  Vladimir Pavlovic,et al.  Face tracking and recognition with visual constraints in real-world videos , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Tal Hassner,et al.  Face recognition in unconstrained videos with matched background similarity , 2011, CVPR 2011.

[31]  Wen Gao,et al.  Manifold-Manifold Distance with application to face recognition based on image set , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[33]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[34]  Simon C. K. Shiu,et al.  Image Set-Based Collaborative Representation for Face Recognition , 2013, IEEE Transactions on Information Forensics and Security.

[35]  Horst Bischof,et al.  Large scale metric learning from equivalence constraints , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Rama Chellappa,et al.  Video-based face recognition via joint sparse representation , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[37]  Likun Huang,et al.  Face recognition based on image sets , 2014 .

[38]  Rama Chellappa,et al.  From sample similarity to ensemble similarity: probabilistic distance measures in reproducing kernel Hilbert space , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Shin'ichi Satoh,et al.  Comparative evaluation of face sequence matching for content-based video access , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[40]  Ajmal S. Mian,et al.  Sparse approximated nearest points for image set classification , 2011, CVPR 2011.

[41]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[42]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[43]  George Karypis,et al.  The Set Classification Problem and Solution Methods , 2008, 2008 IEEE International Conference on Data Mining Workshops.

[44]  WangGang,et al.  Discriminative Multimanifold Analysis for Face Recognition from a Single Training Sample per Person , 2013 .

[45]  Inderjit S. Dhillon,et al.  Information-theoretic metric learning , 2006, ICML '07.

[46]  Hugo Larochelle,et al.  Classification of Sets using Restricted Boltzmann Machines , 2011, UAI.

[47]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[48]  Frédéric Jurie,et al.  PCCA: A new approach for distance learning from sparse pairwise constraints , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.