Manifold Learning by Graduated Optimization

We present an algorithm for manifold learning called manifold sculpting , which utilizes graduated optimization to seek an accurate manifold embedding. An empirical analysis across a wide range of manifold problems indicates that manifold sculpting yields more accurate results than a number of existing algorithms, including Isomap, locally linear embedding (LLE), Hessian LLE (HLLE), and landmark maximum variance unfolding (L-MVU), and is significantly more efficient than HLLE and L-MVU. Manifold sculpting also has the ability to benefit from prior knowledge about expected results.

[1]  Michel Verleysen,et al.  A robust non-linear projection method , 2000, ESANN.

[2]  Martin F. Porter,et al.  An algorithm for suffix stripping , 1997, Program.

[3]  Vijay V. Raghavan,et al.  A critical analysis of vector space model for information retrieval , 1986, J. Am. Soc. Inf. Sci..

[4]  Jeanny Hérault,et al.  Curvilinear component analysis: a self-organizing neural network for nonlinear mapping of data sets , 1997, IEEE Trans. Neural Networks.

[5]  Joshua B. Tenenbaum,et al.  Global Versus Local Methods in Nonlinear Dimensionality Reduction , 2002, NIPS.

[7]  Markus Breitenbach,et al.  Clustering through ranking on manifolds , 2005, ICML '05.

[8]  D. Donoho,et al.  Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[9]  Kilian Q. Weinberger,et al.  Nonlinear Dimensionality Reduction by Semidefinite Programming and Kernel Matrix Factorization , 2005, AISTATS.

[10]  Yoshua Bengio,et al.  Non-Local Manifold Tangent Learning , 2004, NIPS.

[11]  Bernhard Schölkopf,et al.  Kernel Principal Component Analysis , 1997, ICANN.

[12]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[13]  P. Burt.  Fast filter transform for image processing , 1981 .

[14]  Kilian Q. Weinberger,et al.  Learning a kernel matrix for nonlinear dimensionality reduction , 2004, ICML.

[15]  T. C. Edwin Cheng,et al.  An unconstrained optimization problem is NP-hard given an oracle representation of its objective function: a technical note , 2002, Comput. Oper. Res..

[16]  Peter J. Bickel,et al.  Maximum Likelihood Estimation of Intrinsic Dimension , 2004, NIPS.

[17]  Richard Bellman,et al.  Adaptive Control Processes: A Guided Tour , 1961, The Mathematical Gazette.

[18]  Anil K. Jain,et al.  Nonlinear Manifold Learning for Data Stream , 2004, SDM.

[19]  Xin Geng,et al.  Supervised nonlinear dimensionality reduction for visualization and classification , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[20]  Ayhan Demiriz,et al.  Semi-Supervised Clustering Using Genetic Algorithms , 1999 .

[21]  P. J. Burt,et al.  Fast Filter Transforms for Image Processing , 1981 .

[22]  Rudolf Fleischer,et al.  Distance Approximating Dimension Reduction of Riemannian Manifolds , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[23]  Matthew Brand,et al.  Charting a Manifold , 2002, NIPS.

[24]  Yee Leung,et al.  Nonlinear Dimensionality Reduction of Data Lying on the Multicluster Manifold , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[25]  Zhang Yi,et al.  Manifold-Based Learning and Synthesis , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[26]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[27]  H. Hotelling.  Analysis of a complex of statistical variables into principal components. , 1933 .

[28]  Tony R. Martinez,et al.  Iterative Non-linear Dimensionality Reduction with Manifold Sculpting , 2007, NIPS.

[29]  Hongyuan Zha,et al.  A Domain Decomposition Method for Fast Manifold Learning , 2005, NIPS.

[30]  John W. Sammon,et al.  A Nonlinear Mapping for Data Structure Analysis , 1969, IEEE Transactions on Computers.

[31]  Pascal Vincent,et al.  Manifold Parzen Windows , 2002, NIPS.

[32]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[33]  Zhijun Wu,et al.  The Eeective Energy Transformation Scheme as a General Continuation Approach to Global Optimization with Application to Molecular Conformation , 2022 .