LDR-LLE: LLE with Low-Dimensional Neighborhood Representation

The local linear embedding algorithm (LLE) is a non-linear dimension-reducing technique that is widely used for its computational simplicity and intuitive approach. LLE first linearly reconstructs each input point from its nearest neighbors and then preserves these neighborhood relations in a low-dimensional embedding. We show that the reconstruction weights computed by LLE capture the high -dimensional structure of the neighborhoods, and not the low -dimensional manifold structure. Consequently, the weight vectors are highly sensitive to noise. Moreover, this causes LLE to converge to a linear projection of the input, as opposed to its non-linear embedding goal. To resolve both of these problems, we propose to compute the weight vectors using a low-dimensional neighborhood representation. We call this technique LDR-LLE. We present numerical examples of the perturbation and linear projection problems, and of the improved outputs resulting from the low-dimensional neighborhood representation.

[1]  Zhanyi Hu,et al.  The LLE and a linear mapping , 2006, Pattern Recognit..

[2]  Alon Zakai,et al.  Manifold Learning: The Price of Normalization , 2008, J. Mach. Learn. Res..

[3]  Wenbin Chen,et al.  Image denoising through locally linear embedding , 2005, International Conference on Computer Graphics, Imaging and Visualization (CGIV'05).

[4]  Tim W. Nattkemper,et al.  ISOLLE: LLE with geodesic distance , 2006, Neurocomputing.

[5]  Michel Verleysen,et al.  Nonlinear Dimensionality Reduction , 2021, Computer Vision.

[6]  Lawrence K. Saul,et al.  Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifold , 2003, J. Mach. Learn. Res..

[7]  Yoshua Bengio,et al.  Locally Linear Embedding for dimensionality reduction in QSAR , 2004, J. Comput. Aided Mol. Des..

[8]  Wen Gao,et al.  Enhancing Human Face Detection by Resampling Examples Through Manifolds , 2007, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[9]  Wang Xu,et al.  Speech Visualization based on Locally Linear Embedding (LLE) for the Hearing Impaired , 2008, 2008 International Conference on BioMedical Engineering and Informatics.

[10]  Lei Li,et al.  Improved Locally Linear Embedding Through New Distance Computing , 2006, ISNN.

[11]  Matti Pietikäinen,et al.  Efficient Locally Linear Embeddings of Imperfect Manifolds , 2003, MLDM.

[12]  Meng Wang,et al.  SLLE for predicting membrane protein types. , 2005, Journal of theoretical biology.

[13]  Dit-Yan Yeung,et al.  Robust locally linear embedding , 2006, Pattern Recognit..

[14]  Kilian Q. Weinberger,et al.  Unsupervised Learning of Image Manifolds by Semidefinite Programming , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[15]  H. Zha,et al.  Principal manifolds and nonlinear dimensionality reduction via tangent space alignment , 2004, SIAM J. Sci. Comput..

[16]  Gene H. Golub,et al.  Matrix computations , 1983 .

[17]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[18]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[19]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[20]  Jing Wang,et al.  MLLE: Modified Locally Linear Embedding Using Multiple Weights , 2006, NIPS.

[21]  D. Donoho,et al.  Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[22]  Xin Xu,et al.  [A novel method for the determination of redshifts of normal galaxies by non-linear dimensionality reduction]. , 2006, Guang pu xue yu guang pu fen xi = Guang pu.