Robust data representation using locally linear embedding guided PCA

Locally Linear Embedding (LLE) is widely used for embedding data on a nonlinear manifold. It aims to preserve the local neighborhood structure on the data manifold. Our work begins with a new observation that LLE has a natural robustness property. Motivated by this observation, we propose to integrate LLE and PCA into a LLE guided PCA model (LLE-PCA) that incorporates both global structure and local neighborhood structure simultaneously while performs robustly to outliers. LLE-PCA has a compact closed-form solution and can be efficiently computed. Extensive experiments on five datasets show promising results on data reconstruction and improvement on data clustering and semi-supervised learning tasks.

[1]  Xuelong Li,et al.  Joint Embedding Learning and Sparse Regression: A Framework for Unsupervised Feature Selection , 2014, IEEE Transactions on Cybernetics.

[2]  Feiping Nie,et al.  Learning a subspace for clustering via pattern shrinking , 2013, Inf. Process. Manag..

[3]  R. Tibshirani,et al.  Sparse Principal Component Analysis , 2006 .

[4]  Jin Tang,et al.  Graph-Laplacian PCA: Closed-Form Solution and Robustness , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[6]  Feiping Nie,et al.  Nonlinear Dimensionality Reduction with Local Spline Embedding , 2009, IEEE Transactions on Knowledge and Data Engineering.

[7]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[8]  Xuelong Li,et al.  Matrix completion by Truncated Nuclear Norm Regularization , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Feiping Nie,et al.  Orthogonal locality minimizing globality maximizing projections for feature extraction , 2009 .

[10]  Shuicheng Yan,et al.  Latent Low-Rank Representation for subspace segmentation and feature extraction , 2011, 2011 International Conference on Computer Vision.

[11]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[12]  Xin Yu,et al.  A Semi-supervised manifold alignment algorithm and an evaluation method based on local structure preservation , 2017, Neurocomputing.

[13]  Fei Wang,et al.  Label Propagation through Linear Neighborhoods , 2006, IEEE Transactions on Knowledge and Data Engineering.

[14]  Feiping Nie,et al.  A graph regularized dimension reduction method for out-of-sample data , 2017, Neurocomputing.

[15]  Jing Liu,et al.  Robust Structured Subspace Learning for Data Representation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Chris H. Q. Ding,et al.  R1-PCA: rotational invariant L1-norm principal component analysis for robust subspace factorization , 2006, ICML.

[17]  Shuicheng Yan,et al.  Neighborhood preserving embedding , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[18]  Yong Yu,et al.  Robust Subspace Segmentation by Low-Rank Representation , 2010, ICML.

[19]  Feiping Nie,et al.  Learning a subspace for face image clustering via trace ratio criterion , 2009 .

[20]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[22]  H. Zha,et al.  Principal manifolds and nonlinear dimensionality reduction via tangent space alignment , 2004, SIAM J. Sci. Comput..

[23]  Feiping Nie,et al.  Optimal Mean Robust Principal Component Analysis , 2014, ICML.

[24]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[25]  Feiping Nie,et al.  An Iterative Locally Linear Embedding Algorithm , 2012, ICML.

[26]  Bernhard Schölkopf,et al.  A kernel view of the dimensionality reduction of manifolds , 2004, ICML.

[27]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[28]  Ivor W. Tsang,et al.  Flexible Manifold Embedding: A Framework for Semi-Supervised and Unsupervised Dimension Reduction , 2010, IEEE Transactions on Image Processing.

[29]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.