Geometrically local embedding in manifolds for dimension reduction

In this paper, geometrically local embedding (GLE) is presented to discover the intrinsic structure of manifolds as a method in nonlinear dimension reduction. GLE is able to reveal the inner features of the input data in the lower dimension space while suppressing the influence of outliers in the local linear manifold. In addition to feature extraction and representation, GLE behaves as a clustering and classification method by projecting the feature data into low-dimensional separable regions. Through empirical evaluation, the performance of GLE is demonstrated by the visualization of synthetic data in lower dimension, and the comparison with other dimension reduction algorithms with the same data and configuration. Experiments on both pure and noisy data prove the effectiveness of GLE in dimension reduction, feature extraction, data visualization as well as clustering and classification.

[1]  Heiko Hoffmann,et al.  Kernel PCA for novelty detection , 2007, Pattern Recognit..

[2]  Kari Torkkola,et al.  Discriminative features for text document classification , 2003, Formal Pattern Analysis & Applications.

[3]  Yong Wang,et al.  Complete neighborhood preserving embedding for face recognition , 2010, Pattern Recognit..

[4]  Jianmin Jiang,et al.  A Boosted Manifold Learning for Automatic Face Recognition , 2010, Int. J. Pattern Recognit. Artif. Intell..

[5]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[6]  Carl D. Meyer,et al.  Matrix Analysis and Applied Linear Algebra , 2000 .

[7]  Karl Pearson F.R.S..  LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[8]  D. Donoho,et al.  Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[9]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[10]  H. Zha,et al.  Local smoothing for manifold learning , 2004, CVPR 2004.

[11]  Dacheng Tao,et al.  Evolutionary Cross-Domain Discriminative Hessian Eigenmaps , 2010, IEEE Transactions on Image Processing.

[12]  Thomas S. Huang.  Locally Linear Embedded Eigenspace Analysis , 2005 .

[13]  Matti Pietikäinen,et al.  Efficient Locally Linear Embeddings of Imperfect Manifolds , 2003, MLDM.

[14]  Abdullah Al Mamun,et al.  Weighted locally linear embedding for dimension reduction , 2009, Pattern Recognit..

[15]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[16]  Lawrence K. Saul,et al.  Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifold , 2003, J. Mach. Learn. Res..

[17]  Jun Wang,et al.  Reconstruction and analysis of multi-pose face images based on nonlinear dimensionality reduction , 2004, Pattern Recognit..

[18]  Shuzhi Sam Ge,et al.  Neighborhood linear embedding for intrinsic structure discovery , 2008, Machine Vision and Applications.

[19]  Shuzhi Sam Ge,et al.  Feature representation based on intrinsic structure discovery in high dimensional space , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[20]  Steven W. Zucker,et al.  Diffusion Maps and Geometric Harmonics for Automatic Target Recognition (ATR). Volume 2. Appendices , 2007 .

[21]  Dit-Yan Yeung,et al.  Robust locally linear embedding , 2006, Pattern Recognit..

[22]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[23]  John W. Tukey,et al.  A Projection Pursuit Algorithm for Exploratory Data Analysis , 1974, IEEE Transactions on Computers.

[24]  Yiguang Liu,et al.  A novel and quick SVM-based multi-class classifier , 2006, Pattern Recognit..

[25]  G. Stewart.  Perturbation theory for the singular value decomposition , 1990 .

[26]  P. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 1999 .

[27]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[28]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[29]  Shuzhi Sam Ge,et al.  Hand Gesture Recognition and Tracking based on Distributed Locally Linear Embedding , 2006 .