The rare eclipse problem on tiles: Quantised embeddings of disjoint convex sets

Quantised random embeddings are an efficient dimensionality reduction technique which preserves the distances of low-complexity signals up to some controllable additive and multiplicative distortions. In this work, we instead focus on verifying when this technique preserves the separability of two disjoint closed convex sets, i.e., in a quantised view of the “rare eclipse problem” introduced by Bandeira et al. in 2014. This separability would ensure exact classification of signals in such sets from the signatures output by this non-linear dimensionality reduction. We here present a result relating the embedding's dimension, its quantiser resolution and the sets' separation, as well as some numerically testable conditions to illustrate it. Experimental evidence is then provided in the special case of two ℓ2-balls, tracing the phase transition curves that ensure these sets' separability in the embedded domain.

[1]  A. Robert Calderbank,et al.  Bounds on the Number of Measurements for Reliable Compressive Classification , 2016, IEEE Transactions on Signal Processing.

[2]  Y. Gordon.  On Milman's inequality and random subspaces which escape through a mesh in ℝ n , 1988 .

[3]  Sanjoy Dasgupta,et al.  Learning mixtures of Gaussians , 1999, 40th Annual Symposium on Foundations of Computer Science (Cat. No.99CB37039).

[4]  G. Schechtman.  Two observations regarding embedding subsets of Euclidean spaces in normed spaces , 2006 .

[5]  Dustin G. Mixon,et al.  Compressive classification and the rare eclipse problem , 2014, ArXiv.

[6]  W. B. Johnson,et al.  Extensions of Lipschitz mappings into Hilbert space , 1984 .

[7]  Laurent Jacques,et al.  Small width, low distortions: quasi-isometric embeddings with quantized sub-Gaussian random projections , 2015, ArXiv.

[8]  A. Robert Calderbank,et al.  Compressive classification , 2013, 2013 IEEE International Symposium on Information Theory.

[9]  Laurent Jacques,et al.  Consistent Basis Pursuit for Signal and Matrix Estimates in Quantized Compressed Sensing , 2015, IEEE Signal Processing Letters.

[10]  Sanjoy Dasgupta,et al.  An elementary proof of a theorem of Johnson and Lindenstrauss , 2003, Random Struct. Algorithms.

[11]  David L. Neuhoff,et al.  Quantization , 2022, IEEE Trans. Inf. Theory.

[12]  Richard G. Baraniuk,et al.  Signal Processing With Compressive Measurements , 2010, IEEE Journal of Selected Topics in Signal Processing.

[13]  Yaniv Plan,et al.  Dimension Reduction by Random Hyperplane Tessellations , 2014, Discret. Comput. Geom..

[14]  Benjamin Recht,et al.  Near-Optimal Bounds for Binary Embeddings of Arbitrary Sets , 2015, ArXiv.

[15]  Laurent Jacques,et al.  Time for dithering: fast and quantized random embeddings via the restricted isometry property , 2016, ArXiv.

[16]  Laurent Jacques,et al.  Quantization and Compressive Sensing , 2014, ArXiv.

[17]  Benjamin Recht,et al.  Random Features for Large-Scale Kernel Machines , 2007, NIPS.

[18]  G. Pisier.  The volume of convex bodies and Banach space geometry , 1989 .

[19]  Yaniv Plan,et al.  Robust 1-bit Compressed Sensing and Sparse Logistic Regression: A Convex Programming Approach , 2012, IEEE Transactions on Information Theory.

[20]  Laurent Jacques,et al.  A Quantized Johnson–Lindenstrauss Lemma: The Finding of Buffon’s Needle , 2013, IEEE Transactions on Information Theory.

[21]  Joel A. Tropp,et al.  Living on the edge: phase transitions in convex programs with random data , 2013, 1303.6672.

[22]  Dimitris Achlioptas,et al.  Database-friendly random projections: Johnson-Lindenstrauss with binary coins , 2003, J. Comput. Syst. Sci..

[23]  Roman Vershynin,et al.  Introduction to the non-asymptotic analysis of random matrices , 2010, Compressed Sensing.

[24]  Hassan Mansour,et al.  Representation and Coding of Signal Geometry , 2015, ArXiv.

[25]  Laurent Jacques,et al.  Small Width, Low Distortions: Quantized Random Embeddings of Low-Complexity Sets , 2015, IEEE Transactions on Information Theory.