Nonlinear Dimensionality Reduction for Data with Disconnected Neighborhood Graph

Neighborhood graph based nonlinear dimensionality reduction algorithms, such as Isomap and LLE, perform well under an assumption that the neighborhood graph is connected. However, for datasets consisting of multiple clusters or lying on multiple manifolds, the neighborhood graphs are often disconnected, or in other words, have multiple connected components. Neighborhood graph based dimensionality reduction techniques cannot recognize both the local and global properties of such datasets. In this paper, a new method, called enhanced neighborhood graph, is proposed to solve the problem. The concept is to add edges to the neighborhood graph adaptively and iteratively until it becomes connected. Nonlinear dimensionality reduction can then be performed based on the enhanced neighborhood graph. As a result, both local and global properties of the data can be exactly recognized. In this study, thorough simulations on synthetic datasets and natural datasets are conducted. The experimental results corroborate that the proposed method provides significant improvements on dimensionality reduction for data with disconnected neighborhood graph.

[1]  Paul L. Rosin,et al.  Selection of the optimal parameter value for the Isomap algorithm , 2006, Pattern Recognit. Lett..

[2]  Wenbin Li,et al.  Multi-manifold modeling for head pose estimation , 2010, 2010 IEEE International Conference on Image Processing.

[3]  Stéphane Lafon,et al.  Diffusion maps , 2006 .

[4]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.

[5]  Ahmed M. Elgammal,et al.  Learning a Joint Manifold Representation from Multiple Data Sets , 2010, 2010 20th International Conference on Pattern Recognition.

[6]  Shiliang Sun,et al.  Local Tangent Space Discriminant Analysis , 2016, Neural Processing Letters.

[7]  Eric O. Postma,et al.  Dimensionality Reduction: A Comparative Review , 2008 .

[8]  Zhong Jin,et al.  Locally Minimizing Embedding and Globally Maximizing Variance: Unsupervised Linear Difference Projection for Dimensionality Reduction , 2011, Neural Processing Letters.

[9]  Michel Verleysen,et al.  Type 1 and 2 mixtures of Kullback-Leibler divergences as cost functions in dimensionality reduction based on similarity preservation , 2013, Neurocomputing.

[10]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[11]  D. Donoho,et al.  Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[12]  John K. Tsotsos,et al.  Parameterless Isomap with Adaptive Neighborhood Selection , 2006, DAGM-Symposium.

[13]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[14]  Stephen Lin,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  张振跃,et al.  Principal Manifolds and Nonlinear Dimensionality Reduction via Tangent Space Alignment , 2004 .

[16]  Heiko Hoffmann,et al.  Kernel PCA for novelty detection , 2007, Pattern Recognit..

[17]  Volker Blanz,et al.  Component-Based Face Recognition with 3D Morphable Models , 2003, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[18]  Matti Pietikäinen,et al.  Efficient Locally Linear Embeddings of Imperfect Manifolds , 2003, MLDM.

[19]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[20]  Tommy W. S. Chow,et al.  M-Isomap: Orthogonal Constrained Marginal Isomap for Nonlinear Dimensionality Reduction , 2013, IEEE Transactions on Cybernetics.

[21]  Feiping Nie,et al.  A unified framework for semi-supervised dimensionality reduction , 2008, Pattern Recognit..

[22]  Michel Verleysen,et al.  Multi-scale similarities in stochastic neighbour embedding: Reducing dimensionality while preserving both local and global structure , 2015, Neurocomputing.

[23]  Andrés Marino Álvarez-Meza,et al.  Multiple Manifold Learning by Nonlinear Dimensionality Reduction , 2011, CIARP.

[24]  Jeanny Hérault,et al.  Curvilinear component analysis: a self-organizing neural network for nonlinear mapping of data sets , 1997, IEEE Trans. Neural Networks.

[25]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[26]  Michel Verleysen,et al.  Nonlinear Dimensionality Reduction , 2021, Computer Vision.

[27]  Michel Verleysen,et al.  Nonlinear dimensionality reduction of data manifolds with essential loops , 2005, Neurocomputing.

[28]  John W. Sammon,et al.  A Nonlinear Mapping for Data Structure Analysis , 1969, IEEE Transactions on Computers.

[29]  Adil M. Bagirov,et al.  Constrained Self Organizing Maps for Data Clusters Visualization , 2015, Neural Processing Letters.

[30]  Jing Yang,et al.  A Novel Indefinite Kernel Dimensionality Reduction Algorithm: Weighted Generalized Indefinite Kernel Discriminant Analysis , 2013, Neural Processing Letters.

[31]  Kilian Q. Weinberger,et al.  Learning a kernel matrix for nonlinear dimensionality reduction , 2004, ICML.

[32]  B. Nadler,et al.  Diffusion maps, spectral clustering and reaction coordinates of dynamical systems , 2005, math/0503445.

[33]  Michel Verleysen,et al.  Two key properties of dimensionality reduction methods , 2014, 2014 IEEE Symposium on Computational Intelligence and Data Mining (CIDM).

[34]  Lawrence K. Saul,et al.  Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifold , 2003, J. Mach. Learn. Res..

[35]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[36]  Robert E. Tarjan,et al.  Depth-First Search and Linear Graph Algorithms , 1972, SIAM J. Comput..

[37]  Patrick J. F. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 2003 .

[38]  Dong Xu,et al.  Semi-Supervised Dimension Reduction Using Trace Ratio Criterion , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[39]  Hongyuan Zha,et al.  Adaptive Manifold Learning , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Ahmed M. Elgammal,et al.  Learning representations from multiple manifolds , 2016, Pattern Recognit..

[41]  Jia Wei,et al.  Adaptive neighborhood selection for manifold learning , 2008, 2008 International Conference on Machine Learning and Cybernetics.

[42]  Matti Pietikäinen,et al.  Supervised Locally Linear Embedding , 2003, ICANN.

[43]  Daoqiang Zhang,et al.  A New Locality-Preserving Canonical Correlation Analysis Algorithm for Multi-View Dimensionality Reduction , 2013, Neural Processing Letters.

[44]  C. Y. Lee.  An Algorithm for Path Connections and Its Applications , 1961, IRE Trans. Electron. Comput..

[45]  Michel Verleysen,et al.  Nonlinear Dimensionality Reduction for Visualization , 2013, ICONIP.