Universal multi-dimensional scaling

In this paper, we propose a unified algorithmic framework for solving many known variants of MDS. Our algorithm is a simple iterative scheme with guaranteed convergence, and is modular; by changing the internals of a single subroutine in the algorithm, we can switch cost functions and target spaces easily. In addition to the formal guarantees of convergence, our algorithms are accurate; in most cases, they converge to better quality solutions than existing methods in comparable time. Moreover, they have a small memory footprint and scale effectively for large data sets. We expect that this framework will be useful for a number of MDS variants that have not yet been studied. Our framework extends to embedding high-dimensional points lying on a sphere to points on a lower dimensional sphere, preserving geodesic distances. As a complement to this result, we also extend the Johnson-Lindenstrauss Lemma to this spherical setting, by showing that projecting to a random O((1/µ2) log n)-dimensional sphere causes only an eps-distortion in the geodesic distances.

[1]  A. Householder,et al.  Discussion of a set of points in terms of their mutual distances , 1938 .

[2]  Suresh Venkatasubramanian,et al.  The geometric median on Riemannian manifolds with application to robust atlas estimation , 2009, NeuroImage.

[3]  Christos Faloutsos,et al.  FastMap: a fast algorithm for indexing, data-mining and visualization of traditional and multimedia datasets , 1995, SIGMOD '95.

[4]  W. B. Johnson,et al.  Extensions of Lipschitz mappings into Hilbert space , 1984 .

[5]  Patrick Mair,et al.  Multidimensional Scaling Using Majorization: SMACOF in R , 2008 .

[6]  Jirí Matousek,et al.  Low-Distortion Embeddings of Finite Metric Spaces , 2004, Handbook of Discrete and Computational Geometry, 2nd Ed..

[7]  Tamás Sarlós,et al.  Improved Approximation Algorithms for Large Matrices via Random Projections , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[8]  Alexander M. Bronstein,et al.  Numerical Geometry of Non-Rigid Shapes , 2009, Monographs in Computer Science.

[9]  Robert Pless.  Embedding Images in non-Flat Spaces , 2001 .

[10]  R. Gray,et al.  Distortion measures for speech processing , 1980 .

[11]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[12]  H. Karcher.  Riemannian center of mass and mollifier smoothing , 1977 .

[13]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[14]  P. Groenen,et al.  Rank Reduction of Correlation Matrices by Majorization , 2004 .

[15]  I. Norman Katz,et al.  Local convergence in Fermat's problem , 1974, Math. Program..

[16]  P. Groenen,et al.  Modern multidimensional scaling , 1996 .

[17]  Avner Magen,et al.  Dimensionality Reductions That Preserve Volumes and Distance to Affine Spaces, and Their Algorithmic Applications , 2002, RANDOM.

[18]  Kaizhong Zhang,et al.  Evaluating a class of distance-mapping algorithms for data mining and clustering , 1999, KDD '99.

[19]  Ron Kimmel,et al.  Texture Mapping via Spherical Multi-dimensional Scaling , 2005, Scale-Space.

[20]  Joshua B. Tenenbaum,et al.  Global Versus Local Methods in Nonlinear Dimensionality Reduction , 2002, NIPS.

[21]  Lawrence M. Ostresh.  On the Convergence of a Class of Iterative Methods for Solving the Weber Location Problem , 1978, Oper. Res..

[22]  J. Leeuw.  Applications of Convex Analysis to Multidimensional Scaling , 2000 .

[23]  Myron Wish,et al.  Three-Way Multidimensional Scaling , 1978 .

[24]  Thorsten Joachims,et al.  Learning to classify text using support vector machines - methods, theory and algorithms , 2002, The Kluwer international series in engineering and computer science.

[25]  Pankaj K. Agarwal,et al.  Embeddings of surfaces, curves, and moving points in euclidean space , 2007, SCG '07.

[26]  Naftali Tishby,et al.  Distributional Clustering of English Words , 1993, ACL.

[27]  Prosenjit Bose,et al.  Fast approximations for sums of distances, clustering and the Fermat-Weber problem , 2003, Computational geometry.

[28]  Trevor F. Cox,et al.  Multidimensional Scaling, Second Edition , 2000 .

[29]  A. Buja,et al.  Local Multidimensional Scaling for Nonlinear Dimension Reduction, Graph Drawing, and Proximity Analysis , 2009 .

[30]  Sanjoy Dasgupta,et al.  Robust Euclidean embedding , 2006, ICML.

[31]  W. Torgerson.  Multidimensional scaling: I. Theory and method , 1952 .

[32]  E. Aronson,et al.  Theory and method , 1985 .

[33]  Harold W. Kuhn,et al.  A note on Fermat's problem , 1973, Math. Program..

[34]  M. Shirosaki.  Another proof of the defect relation for moving targets , 1991 .

[35]  J. Kruskal.  Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis , 1964 .

[36]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[37]  Ulrik Brandes,et al.  Eigensolver Methods for Progressive Multidimensional Scaling of Large Data , 2006, GD.

[38]  Jack Brimberg,et al.  Global Convergence of a Generalized Iterative Procedure for the Minisum Location Problem with lp Distances , 1993, Oper. Res..

[39]  I. Olkin,et al.  Inequalities: Theory of Majorization and Its Applications , 1980 .