Isometric sketching of any set via the Restricted Isometry Property

In this paper we show that for the purposes of dimensionality reduction certain class of structured random matrices behave similarly to random Gaussian matrices. This class includes several matrices for which matrix-vector multiply can be computed in log-linear time, providing efficient dimensionality reduction of general sets. In particular, we show that using such matrices any set from high dimensions can be embedded into lower dimensions with near optimal distortion. We obtain our results by connecting dimensionality reduction of any set to dimensionality reduction of sparse vectors via a chaining argument.

[1]  W. B. Johnson,et al.  Extensions of Lipschitz mappings into Hilbert space , 1984 .

[2]  Y. Gordon.  On Milman's inequality and random subspaces which escape through a mesh in ℝ n , 1988 .

[3]  Peter Frankl,et al.  The Johnson-Lindenstrauss lemma and the sphericity of some graphs , 1987, J. Comb. Theory B.

[4]  Dimitris Achlioptas,et al.  Database-friendly random projections: Johnson-Lindenstrauss with binary coins , 2003, J. Comput. Syst. Sci..

[5]  Sanjoy Dasgupta,et al.  An elementary proof of a theorem of Johnson and Lindenstrauss , 2003, Random Struct. Algorithms.

[6]  S. Mendelson,et al.  Empirical processes and random projections , 2005 .

[7]  M. Talagrand.  The Generic chaining : upper and lower bounds of stochastic processes , 2005 .

[8]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[9]  Tamás Sarlós,et al.  Improved Approximation Algorithms for Large Matrices via Random Projections , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[10]  M. Rudelson,et al.  Sparse reconstruction by convex relaxation: Fourier and Gaussian measurements , 2006, 2006 40th Annual Conference on Information Sciences and Systems.

[11]  S. Mendelson,et al.  Reconstruction and Subgaussian Operators in Asymptotic Geometric Analysis , 2007 .

[12]  Trac D. Tran,et al.  Fast and efficient dimensionality reduction using Structurally Random Matrices , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[13]  Daniel M. Kane,et al.  A Derandomized Sparse Johnson-Lindenstrauss Transform , 2010, Electron. Colloquium Comput. Complex..

[14]  Rachel Ward,et al.  New and Improved Johnson-Lindenstrauss Embeddings via the Restricted Isometry Property , 2010, SIAM J. Math. Anal..

[15]  Nir Ailon,et al.  An almost optimal unrestricted fast Johnson-Lindenstrauss transform , 2010, SODA '11.

[16]  Amit Singer,et al.  Dense Fast Random Projections and Lean Walsh Transforms , 2008, APPROX-RANDOM.

[17]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[18]  Roman Vershynin,et al.  Introduction to the non-asymptotic analysis of random matrices , 2010, Compressed Sensing.

[19]  Pablo A. Parrilo,et al.  The Convex Geometry of Linear Inverse Problems , 2010, Foundations of Computational Mathematics.

[20]  R. Vershynin.  Lectures in Geometric Functional Analysis , 2012 .

[21]  Venkatesan Guruswami,et al.  Restricted Isometry of Fourier Matrices and List Decodability of Random Linear Codes , 2012, SIAM J. Comput..

[22]  Michael B. Wakin,et al.  Stable Manifold Embeddings With Structured Random Matrices , 2012, IEEE Journal of Selected Topics in Signal Processing.

[23]  V. Koltchinskii,et al.  Bounding the smallest singular value of a random matrix without concentration , 2013, 1312.3580.

[24]  S. Foucart,et al.  Random Sampling in Bounded Orthonormal Systems , 2013 .

[25]  Sjoerd Dirksen,et al.  Tail bounds via generic chaining , 2013, ArXiv.

[26]  J. Tropp.  Convex recovery of a structured signal from independent random linear measurements , 2014, ArXiv.

[27]  Nir Ailon,et al.  Fast and RIP-Optimal Transforms , 2013, Discrete & Computational Geometry.

[28]  Martin J. Wainwright,et al.  Randomized sketches of convex programs with sharp guarantees , 2014, 2014 IEEE International Symposium on Information Theory.

[29]  Mary Wootters,et al.  New constructions of RIP matrices with fast multiplication and fewer rows , 2012, SODA.

[30]  Shahar Mendelson,et al.  Learning without Concentration , 2014, COLT.

[31]  J. Bourgain.  An Improved Estimate in the Restricted Isometry Problem , 2014 .

[32]  Sjoerd Dirksen,et al.  Toward a unified theory of sparse dimensionality reduction in Euclidean space , 2013, STOC.

[33]  Sjoerd Dirksen,et al.  Dimensionality Reduction with Subgaussian Matrices: A Unified Theory , 2014, Foundations of Computational Mathematics.

[34]  Oded Regev,et al.  The Restricted Isometry Property of Subsampled Fourier Matrices , 2015, SODA.