Supervised locally linear embedding algorithm based on orthogonal matching pursuit

Supervised locally linear embedding (SLLE) has been proposed for classification tasks. SLLE can take full use of the label information and select neighbours only in the same class. However, SLLE uses the least squares (LSs) method for solving a set of linear equations to obtain linear representation coefficients, which relates to the inverse of a matrix. If the matrix is singular, the solution to the set of linear equations does not exist. Additionally, if the size of neighbourhood is not appropriate, some further neighbours along the manifold would be selected. To remedy those, this study deals with SLLE based on orthogonal matching pursuit (SLLE-OMP) by introducing OMP into SLLE. In SLLE-OMP, LS is replaced by OMP and OMP can reselect new neighbours from old ones. Experimental results on some real-world datasets show that SLLE-OMP can achieve better classification performance compared with SLLE.

[1]  Hongyuan Zha,et al.  Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment , 2002, ArXiv.

[2]  Li Zhang,et al.  On the sparseness of 1-norm support vector machines , 2010, Neural Networks.

[3]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[4]  Matti Pietikäinen,et al.  Incremental locally linear embedding , 2005, Pattern Recognit..

[5]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[6]  Ronald A. DeVore,et al.  Deterministic constructions of compressed sensing matrices , 2007, J. Complex..

[7]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[8]  D. D. Ridder,et al.  Locally linear embedding for classification , 2002 .

[9]  Lawrence K. Saul,et al.  Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifold , 2003, J. Mach. Learn. Res..

[10]  Jean-Luc Starck,et al.  Sparse Solution of Underdetermined Systems of Linear Equations by Stagewise Orthogonal Matching Pursuit , 2012, IEEE Transactions on Information Theory.

[11]  D. Donoho,et al.  Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[12]  M. Loog,et al.  Local Fisher embedding , 2004, ICPR 2004.

[13]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[14]  Sheng Chen,et al.  Orthogonal least squares methods and their application to non-linear system identification , 1989 .

[15]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[16]  Hongyuan Zha,et al.  Principal manifolds and nonlinear dimensionality reduction via tangent space alignment , 2004, SIAM J. Sci. Comput..

[17]  R.G. Baraniuk,et al.  Compressive Sensing [Lecture Notes] , 2007, IEEE Signal Processing Magazine.

[18]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[19]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[20]  Feiping Nie,et al.  An Iterative Locally Linear Embedding Algorithm , 2012, ICML.

[21]  David J. Kriegman,et al.  Nine points of light: acquiring subspaces for face recognition under variable lighting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[22]  T. Poggio,et al.  Prediction of central nervous system embryonal tumour outcome based on gene expression , 2002, Nature.