PARAMAP vs. Isomap: A Comparison of Two Nonlinear Mapping Algorithms

Dimensionality reduction techniques are used for representing higher dimensional data by a more parsimonious and meaningful lower dimensional structure. In this paper we will study two such approaches, namely Carroll’s Parametric Mapping (abbreviated PARAMAP) (Shepard and Carroll, 1966) and Tenenbaum’s Isometric Mapping (abbreviated Isomap) (Tenenbaum, de Silva, and Langford, 2000). The former relies on iterative minimization of a cost function while the latter applies classical MDS after a preprocessing step involving the use of a shortest path algorithm to define approximate geodesic distances. We will develop a measure of congruence based on preservation of local structure between the input data and the mapped low dimensional embedding, and compare the different approaches on various sets of data, including points located on the surface of a sphere, some data called the "Swiss Roll data", and truncated spheres.

[1]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[2]  Andrew R. Webb,et al.  Multidimensional scaling by iterative majorization using radial basis functions , 1995, Pattern Recognit..

[3]  Paul Scheunders,et al.  Non-linear dimensionality reduction techniques for unsupervised feature extraction , 1998, Pattern Recognit. Lett..

[4]  M. J. D. Powell,et al.  Restart procedures for the conjugate gradient method , 1977, Math. Program..

[5]  R. H. Kent,et al.  The Mean Square Successive Difference , 1941 .

[6]  Michel Verleysen,et al.  Curvilinear Distance Analysis versus Isomap , 2002, ESANN.

[7]  T. C. Hu.  Revised Matrix Algorithms for Shortest Paths , 1967 .

[8]  John W. Sammon,et al.  A Nonlinear Mapping for Data Structure Analysis , 1969, IEEE Transactions on Computers.

[9]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[10]  Mukund Balasubramanian,et al.  The Isomap Algorithm and Topological Stability , 2002, Science.

[11]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[12]  William M. Rand,et al.  Objective Criteria for the Evaluation of Clustering Methods , 1971 .

[13]  Roderick P. McDonald.  Factor interaction in nonlinear factor analysis. , 1967 .

[14]  R. P. McDonald,et al.  A general approach to nonlinear factor analysis , 1962 .

[15]  Kurt Hornik,et al.  Neural networks and principal component analysis: Learning from examples without local minima , 1989, Neural Networks.

[16]  R. Shepard,et al.  A nonmetric variety of linear factor analysis , 1974 .

[17]  J. Kruskal.  Nonmetric multidimensional scaling: A numerical method , 1964 .

[18]  Joseph L. Zinnes,et al.  Theory and Methods of Scaling. , 1958 .

[19]  Michel Verleysen,et al.  Nonlinear projection with curvilinear distances: Isomap versus curvilinear distance analysis , 2004, Neurocomputing.

[20]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[21]  M. Kramer.  Nonlinear principal component analysis using autoassociative neural networks , 1991 .

[22]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[23]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[24]  Ian T. Foster,et al.  Designing and building parallel programs - concepts and tools for parallel software engineering , 1995 .

[25]  Joshua B. Tenenbaum,et al.  Global Versus Local Methods in Nonlinear Dimensionality Reduction , 2002, NIPS.

[26]  J. Neumann.  Distribution of the Ratio of the Mean Square Successive Difference to the Variance , 1941 .

[27]  Edsger W. Dijkstra,et al.  A note on two problems in connexion with graphs , 1959, Numerische Mathematik.

[28]  George Karypis,et al.  Introduction to Parallel Computing , 1994 .

[29]  R. P. McDonald,et al.  Nonlinear factor analysis. , 1967 .