A supervised non-linear dimensionality reduction approach for manifold learning

In this paper we introduce a novel supervised manifold learning technique called Supervised Laplacian Eigenmaps (S-LE), which makes use of class label information to guide the procedure of non-linear dimensionality reduction by adopting the large margin concept. The graph Laplacian is split into two components: within-class graph and between-class graph to better characterize the discriminant property of the data. Our approach has two important characteristics: (i) it adaptively estimates the local neighborhood surrounding each sample based on data density and similarity and (ii) the objective function simultaneously maximizes the local margin between heterogeneous samples and pushes the homogeneous samples closer to each other. Our approach has been tested on several challenging face databases and it has been conveniently compared with other linear and non-linear techniques, demonstrating its superiority. Although we have concentrated in this paper on the face recognition problem, the proposed approach could also be applied to other category of objects characterized by large variations in their appearance (such as hand or body pose, for instance).

[1]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[2]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[3]  C. Viroli,et al.  Supervised locally linear embedding for classification : an application to gene expression data analysis Supervised locally linear embedding in problemi di classificazione : un ’ applicazione all ’ analisi di dati di espressione genica , 2005 .

[4]  M. Naderi.  Think globally... , 2004, HIV prevention plus!.

[5]  Kilian Q. Weinberger,et al.  Spectral Methods for Dimensionality Reduction , 2006, Semi-Supervised Learning.

[6]  Xin Geng,et al.  Supervised nonlinear dimensionality reduction for visualization and classification , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[7]  Daoqiang Zhang,et al.  Efficient and robust feature extraction by maximum margin criterion , 2003, IEEE Transactions on Neural Networks.

[8]  Josef Kittler,et al.  Locally linear discriminant analysis for multimodally distributed classes for face recognition with a single model image , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Amir Globerson,et al.  Metric Learning by Collapsing Classes , 2005, NIPS.

[10]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Ali Ghodsi,et al.  Distance metric learning vs. Fisher discriminant analysis , 2008, AAAI 2008.

[12]  Dewen Hu,et al.  Incremental Laplacian eigenmaps by preserving adjacent information between data points , 2009, Pattern Recognit. Lett..

[13]  Shuicheng Yan,et al.  Neighborhood preserving embedding , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[14]  Lide Wu,et al.  Face recognition by stepwise nonparametric margin maximum criterion , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[15]  David G. Stork,et al.  Pattern Classification , 1973 .

[16]  Patrick J. F. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 2003 .

[17]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[18]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[19]  Limei Zhang,et al.  Graph-optimized locality preserving projections , 2010, Pattern Recognit..

[20]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Jianbo Shi,et al.  Multiclass spectral clustering , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[22]  Lawrence K. Saul,et al.  Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifold , 2003, J. Mach. Learn. Res..

[23]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[24]  Shuicheng Yan,et al.  Graph embedding: a general framework for dimensionality reduction , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[25]  Kilian Q. Weinberger,et al.  Unsupervised Learning of Image Manifolds by Semidefinite Programming , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[26]  Daoqiang Zhang,et al.  marginFace: A novel face recognition method by average neighborhood margin maximization , 2009, Pattern Recognit..

[27]  Xiaolong Teng,et al.  Face recognition using discriminant locality preserving projections , 2006, Image Vis. Comput..

[28]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[29]  Matti Pietikäinen,et al.  Supervised Locally Linear Embedding Algorithm for Pattern Recognition , 2003, IbPRIA.

[30]  Shuicheng Yan,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007 .

[31]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[32]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[33]  Jian Yang,et al.  LPP solution schemes for use with face recognition , 2010, Pattern Recognit..

[34]  Xuelong Li,et al.  Patch Alignment for Dimensionality Reduction , 2009, IEEE Transactions on Knowledge and Data Engineering.

[35]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[36]  Ying Wu,et al.  Locality Versus Globality: Query-Driven Localized Linear Models for Facial Image Computing , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[37]  D. Donoho,et al.  Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[38]  Minping Jia,et al.  Supervised Laplacian Eigenmaps for Machinery Fault Classification , 2009, 2009 WRI World Congress on Computer Science and Information Engineering.

[39]  George Bebis,et al.  Face recognition experiments with random projection , 2005, SPIE Defense + Commercial Sensing.

[40]  Jian Yang,et al.  Quotient vs. difference: Comparison between the two discriminant criteria , 2010, Neurocomputing.