A Nonlinear Approach to Dimension Reduction

AbstractThe $$\ell _2$$ℓ2 flattening lemma of Johnson and Lindenstrauss (in: Proceedings of the conference in modern analysis and probability, 1984) is a powerful tool for dimension reduction. It has been conjectured that the target dimension bounds can be refined and bounded in terms of the intrinsic dimensionality of the dataset (for example, the doubling dimension). One such problem was proposed by Lang and Plaut (Geom Dedicata 87(1–3):285–307, 2001) (see also Abraham et al. in: Proceedings of the 20th annual ACM–SIAM symposium on discrete algorithms, 2008; Chan et al. in: J ACM 57(4):1–26, 2010; Gupta et al. in: Proceedings of the 44th annual IEEE symposium on foundations of computer science, 2003; Matoušek in: Open problems on low-distortion embeddings of finite metric spaces, 2002), and is still open. We prove another result in this line of work: The snowflake metric $$d^\alpha $$dα ($$\alpha <1$$α<1) of a doubling set $$S\subset \ell _2$$S⊂ℓ2 embeds with constant distortion into $$\ell _2^D$$ℓ2D for dimension D that depends solely on the doubling constant of the metric. In fact, the distortion can be made arbitrarily close to 1, and the target dimension is polylogarithmic in the doubling constant. Our techniques are robust and extend to the more difficult space $$\ell _1$$ℓ1, although the dimension bounds here are quantitatively inferior to those for $$\ell _2$$ℓ2.

[1]  Ittai Abraham,et al.  On low dimensional local embeddings , 2009, SODA.

[2]  Kenneth Ward Church,et al.  Nonlinear Estimators and Tail Bounds for Dimension Reduction in l1 Using Cauchy Random Projections , 2006, J. Mach. Learn. Res..

[3]  P. Assouad.  Plongements lipschitziens dans Rn , 2003 .

[4]  M. Talagrand.  Embedding Subspaces of L 1 into l N 1 , 1990 .

[5]  Anupam Gupta,et al.  Ultra-low-dimensional embeddings for doubling metrics , 2008, SODA '08.

[6]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[7]  U. Lang,et al.  Bilipschitz Embeddings of Metric Spaces into Space Forms , 2001 .

[8]  Dana Ron,et al.  Testing metric properties , 2003, Inf. Comput..

[9]  Mihai Badoiu,et al.  Approximation algorithms for low-distortion embeddings into low-dimensional spaces , 2005, SODA '05.

[10]  Piotr Indyk,et al.  Low-distortion embeddings of general metrics into the line , 2005, STOC '05.

[11]  James R. Lee,et al.  Metric Structures in L1: Dimension, Snowflakes, and Average Distortion , 2004, LATIN.

[12]  J. R. Lee,et al.  Embedding the diamond graph in Lp and dimension reduction in L1 , 2004, math/0407520.

[13]  I. J. Schoenberg.  Metric spaces and completely monotone functions , 1938 .

[14]  M. Talagrand.  Embedding subspaces of ₁ into ^{}₁ , 1990 .

[15]  B. Recht,et al.  A Nash-type Dimensionality Reduction for Discrete Subsets of L 2 , 2008 .

[16]  Robert Krauthgamer,et al.  Measured descent: a new embedding method for finite metrics , 2004, 45th Annual IEEE Symposium on Foundations of Computer Science.

[17]  Keith Ball.  Isometric Embedding in lp-spaces , 1990, Eur. J. Comb..

[18]  P. Assouad.  Plongements lipschitziens dans ${\bbfR}\sp n$ , 1983 .

[19]  Michel Deza,et al.  Geometry of cuts and metrics , 2009, Algorithms and combinatorics.

[20]  Gideon Schechtman,et al.  Factorizations of natural embeddings of $l^{n}_{p}$ into $L_{r}$, I , 1988 .

[21]  Jirí Matousek,et al.  Inapproximability for Metric Embeddings into R^d , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[22]  Leonard J. Schulman,et al.  Clustering for edge-cost minimization (extended abstract) , 2000, STOC '00.

[23]  Robert Krauthgamer,et al.  Navigating nets: simple algorithms for proximity search , 2004, SODA '04.

[24]  Ori Sasson,et al.  Property testing of data dimensionality , 2003, SODA '03.

[25]  Ittai Abraham,et al.  Embedding metric spaces in their intrinsic dimension , 2008, SODA '08.

[26]  Jirí Matousek,et al.  Low-Distortion Embeddings of Finite Metric Spaces , 2004, Handbook of Discrete and Computational Geometry, 2nd Ed..

[27]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[28]  Yair Bartal,et al.  Dimension reduction for ultrametrics , 2004, SODA '04.

[29]  G. Schechtman,et al.  Factorizations of natural embeddings of l n p into Lr, II by , 1992 .

[30]  Sunil Arya,et al.  Space-time tradeoffs for approximate nearest neighbor searching , 2009, JACM.

[31]  M. Talagrand.  Approximating a helix in finitely many dimensions , 1992 .

[32]  J. Matousek,et al.  On the distortion required for embedding finite metric spaces into normed spaces , 1996 .

[33]  James R. Lee,et al.  Metric structures in L1: dimension, snowflakes, and average distortion , 2005, Eur. J. Comb..

[34]  M. Talagrand.  Embedding subspaces of L1 into l1N , 1990 .

[35]  Sanjoy Dasgupta,et al.  Random projection trees and low dimensional manifolds , 2008, STOC.

[36]  J. Matousek.  INAPPROXIMABILITY FOR METRIC EMBEDDINGS INTO R , 2010 .

[37]  Leonard J. Schulman,et al.  Dimensionality reduction: beyond the Johnson-Lindenstrauss bound , 2011, SODA '11.

[38]  W. B. Johnson,et al.  Extensions of Lipschitz mappings into Hilbert space , 1984 .

[39]  Sariel Har-Peled,et al.  Fast construction of nets in low dimensional metrics, and their applications , 2004, SCG.

[40]  Noga Alon,et al.  Problems and results in extremal combinatorics--I , 2003, Discret. Math..

[41]  Rafail Ostrovsky,et al.  Polynomial-time approximation schemes for geometric min-sum median clustering , 2002, JACM.

[42]  Robert Krauthgamer,et al.  Bounded geometries, fractals, and low-distortion embeddings , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[43]  Piotr Indyk,et al.  Stable distributions, pseudorandom generators, embeddings, and data stream computation , 2006, JACM.

[44]  Patrice Assouad.  Plongements lipschitziens dans ${\mathbb {R}}^n$ , 1983 .

[45]  M. Talagrand.  Embedding Subspaces of L p in ℓ p N , 1995 .

[46]  G. Schechtman.  More on embedding subspaces of $L_p$ in $l^n_r$ , 1987 .

[47]  M. D. Kirszbraun.  Über die zusammenziehende und Lipschitzsche Transformationen , 1934 .

[48]  I. J. Schoenberg,et al.  Metric spaces and positive definite functions , 1938 .

[49]  Gideon Schechtman,et al.  Factorizations of natural embeddings of $l^n_p$ into $L_r$. II. , 1991 .

[50]  Hui Zhang,et al.  Predicting Internet network distance with coordinates-based approaches , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[51]  Assaf Naor,et al.  The Johnson–Lindenstrauss Lemma Almost Characterizes Hilbert Space, But Not Quite , 2008, SODA.

[52]  Piotr Indyk,et al.  Nearest-neighbor-preserving embeddings , 2007, TALG.