Compressive classification and the rare eclipse problem

This paper addresses the fundamental question of when convex sets remain disjoint after random projection. We provide an analysis using ideas from high-dimensional convex geometry. For ellipsoids, we provide a bound in terms of the distance between these ellipsoids and simple functions of their polynomial coefficients. As an application, this theorem provides bounds for compressive classification of convex sets. Rather than assuming that the data to be classified is sparse, our results show that the data can be acquired via very few measurements yet will remain linearly separable. We demonstrate the feasibility of this approach in the context of hyperspectral imaging.

[1]  T. L. Reguianski.  The Air Force Institute of Technology , 1962 .

[2]  J. Kuelbs.  Probability on Banach spaces , 1978 .

[3]  W. B. Johnson,et al.  Extensions of Lipschitz mappings into Hilbert space , 1984 .

[4]  Y. Gordon.  On Milman's inequality and random subspaces which escape through a mesh in ℝ n , 1988 .

[5]  Jon M. Kleinberg,et al.  Two algorithms for nearest-neighbor search in high dimensions , 1997, STOC '97.

[6]  Piotr Indyk,et al.  Approximate nearest neighbors: towards removing the curse of dimensionality , 1998, STOC '98.

[7]  Sanjoy Dasgupta,et al.  Learning mixtures of Gaussians , 1999, 40th Annual Symposium on Foundations of Computer Science (Cat. No.99CB37039).

[8]  Sanjoy Dasgupta,et al.  Experiments with Random Projection , 2000, UAI.

[9]  Sanjoy Dasgupta,et al.  An elementary proof of a theorem of Johnson and Lindenstrauss , 2003, Random Struct. Algorithms.

[10]  G. Paouris.  Concentration of mass on convex bodies , 2006 .

[11]  J. Haupt,et al.  Compressive Sampling for Signal Classification , 2006, 2006 Fortieth Asilomar Conference on Signals, Systems and Computers.

[12]  S.A. Robila,et al.  NEW APPROACHES FOR FEATURE EXTRACTION IN HYPERSPECTRAL IMAGERY , 2006, 2006 IEEE Long Island Systems, Applications and Technology Conference.

[13]  Richard G. Baraniuk,et al.  Multiscale Random Projections for Compressive Classification , 2007, 2007 IEEE International Conference on Image Processing.

[14]  Richard G. Baraniuk,et al.  The smashed filter for compressive classification and target recognition , 2007, Electronic Imaging.

[15]  Richard G. Baraniuk,et al.  1-Bit compressive sensing , 2008, 2008 42nd Annual Conference on Information Sciences and Systems.

[16]  Rabab Kreidieh Ward,et al.  Robust Classifiers for Data Reduced via Random Projections , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[17]  Robert D. Nowak,et al.  Sample complexity for 1-bit compressed sensing and sparse classification , 2010, 2010 IEEE International Symposium on Information Theory.

[18]  Yehuda Pfeffer.  A Micro-Mirror Array based System for Compressive Sensing of Hyperspectral Data , 2010 .

[19]  Yaniv Plan,et al.  One‐Bit Compressed Sensing by Linear Programming , 2011, ArXiv.

[20]  Pablo A. Parrilo,et al.  The Convex Geometry of Linear Inverse Problems , 2010, Foundations of Computational Mathematics.

[21]  Robert J. Johnson.  Improved Feature Extraction, Feature Selection, and Identification Techniques that Create a Fast Unsupervised Hyperspectral Target Detection Algorithm , 2012 .

[22]  Kamelia Aryafar.  Music Genre Classification Using Sparsity-Eager Support Vector Machines , 2012 .

[23]  Ali Shokoufandeh,et al.  Automatic musical genre classification using sparsity-eager support vector machines , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[24]  Joel A. Tropp,et al.  Living on the edge: A geometric theory of phase transitions in convex optimization , 2013, ArXiv.

[25]  Lawrence Carin,et al.  Coded Hyperspectral Imaging and Blind Compressive Sensing , 2013, SIAM J. Imaging Sci..

[26]  Joel A. Tropp,et al.  Living on the edge: phase transitions in convex programs with random data , 2013, 1303.6672.

[27]  A. Robert Calderbank,et al.  Compressive Classification of a Mixture of Gaussians: Analysis, Designs and Geometrical Interpretation , 2014, ArXiv.

[28]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[29]  Noga Alon,et al.  Linear Boolean Classification, Coding and the Critical Problem , 2016, IEEE Transactions on Information Theory.