Nonlinear Dimensionality Reduction via Path-Based Isometric Mapping

Nonlinear dimensionality reduction methods have demonstrated top-notch performance in many pattern recognition and image classification tasks. Despite their popularity, they suffer from highly expensive time and memory requirements, which render them inapplicable to large-scale datasets. To leverage such cases we propose a new method called “Path-Based Isomap”. Similar to Isomap, we exploit geodesic paths to find the low-dimensional embedding. However, instead of preserving pairwise geodesic distances, the low-dimensional embedding is computed via a path-mapping algorithm. Due to the much fewer number of paths compared to number of data points, a significant improvement in time and memory complexity with a comparable performance is achieved. The method demonstrates state-of-the-art performance on well-known synthetic and real-world datasets, as well as in the presence of noise.

[1]  Stéphane Lafon,et al.  Diffusion maps , 2006 .

[2]  Joshua B. Tenenbaum,et al.  Sparse multidimensional scaling using land-mark points , 2004 .

[3]  Joshua B. Tenenbaum,et al.  Global Versus Local Methods in Nonlinear Dimensionality Reduction , 2002, NIPS.

[4]  M. V. Velzen,et al.  Self-organizing maps , 2007 .

[5]  Sridhar Mahadevan,et al.  Manifold alignment using Procrustes analysis , 2008, ICML '08.

[6]  Stefano Pallottino,et al.  Shortest-path methods: Complexity, interrelations and new propositions , 1984, Networks.

[7]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[8]  Ameet Talwalkar,et al.  Large-scale manifold learning , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Piotr Indyk,et al.  Approximate nearest neighbors: towards removing the curse of dimensionality , 1998, STOC '98.

[10]  Li Zhuo,et al.  A comparative study of dimensionality reduction methods for large-scale image retrieval , 2014, Neurocomputing.

[11]  Luc Van Gool,et al.  Iterative Nearest Neighbors for classification and dimensionality reduction , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[13]  Eric O. Postma,et al.  Dimensionality Reduction: A Comparative Review , 2008 .

[14]  Piotr Indyk,et al.  Nearest Neighbors in High-Dimensional Spaces , 2004, Handbook of Discrete and Computational Geometry, 2nd Ed..

[15]  Kilian Q. Weinberger,et al.  An Introduction to Nonlinear Dimensionality Reduction by Maximum Variance Unfolding , 2006, AAAI.

[16]  T. Gneiting,et al.  Estimators of Fractal Dimension : Assessing the Roughness of Time Series and Spatial Data , 2010 .

[17]  Jianzhong Wang.  Local Tangent Space Alignment , 2012 .

[18]  D. Donoho,et al.  Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[19]  Lawrence K. Saul,et al.  Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifold , 2003, J. Mach. Learn. Res..

[20]  Alexander M. Bronstein,et al.  Nonlinear Dimensionality Reduction by Topologically Constrained Isometric Embedding , 2010, International Journal of Computer Vision.

[21]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[22]  Guodong Guo,et al.  Simultaneous dimensionality reduction and human age estimation via kernel partial least squares regression , 2011, CVPR 2011.

[23]  E. Henry,et al.  [8] Singular value decomposition: Application to analysis of experimental data , 1992 .

[24]  James T. Kwok,et al.  Large-Scale Nyström Kernel Matrix Approximation Using Randomized SVD , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Bernard Fertil,et al.  RankVisu: Mapping from the neighborhood network , 2009, Neurocomputing.

[26]  Shimon Even,et al.  Graph Algorithms: Contents , 2011 .

[27]  Søren Hauberg,et al.  Model Transport: Towards Scalable Transfer Learning on Manifolds , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Shimon Even,et al.  Graph Algorithms , 1979 .

[29]  Hongyuan Zha,et al.  Spectral Properties of the Alignment Matrices in Manifold Learning , 2009, SIAM Rev..

[30]  Richard G. Baraniuk,et al.  Random Projections of Smooth Manifolds , 2009, Found. Comput. Math..

[31]  Aswin C. Sankaranarayanan,et al.  Learning Manifolds in the Wild , 2012 .

[32]  Patrick J. F. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 2003 .

[33]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[34]  John Platt,et al.  FastMap, MetricMap, and Landmark MDS are all Nystrom Algorithms , 2005, AISTATS.

[35]  张振跃,et al.  Principal Manifolds and Nonlinear Dimensionality Reduction via Tangent Space Alignment , 2004 .

[36]  Hans-Peter Kriegel,et al.  Optimal multi-step k-nearest neighbor search , 1998, SIGMOD '98.

[37]  Geoffrey E. Hinton,et al.  Stochastic Neighbor Embedding , 2002, NIPS.

[38]  R. Osserman.  The isoperimetric inequality , 1978 .

[39]  Donald B. Percival,et al.  Estimators of Fractal Dimension: Assessing the Roughness of Time Series and Spatial Data , 2011, 1101.1444.

[40]  I. Hassan.  Embedded , 2005, The Cyber Security Handbook.

[41]  Mukund Balasubramanian,et al.  The Isomap Algorithm and Topological Stability , 2002, Science.

[42]  Xian-Sheng Hua,et al.  Ensemble Manifold Regularization , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Michael W. Berry,et al.  Large-Scale Sparse Singular Value Computations , 1992 .

[44]  Melba M. Crawford,et al.  Manifold-Learning-Based Feature Extraction for Classification of Hyperspectral Data: A Review of Advances in Manifold Learning , 2014, IEEE Signal Processing Magazine.

[45]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[46]  Cheng Li,et al.  Fisher Linear Discriminant Analysis , 2014 .

[47]  Jeanny Hérault,et al.  Curvilinear component analysis: a self-organizing neural network for nonlinear mapping of data sets , 1997, IEEE Trans. Neural Networks.