Dimensionality reduction via compressive sensing

Compressive sensing is an emerging field predicated upon the fact that, if a signal has a sparse representation in some basis, then it can be almost exactly reconstructed from very few random measurements. Many signals and natural images, for example under the wavelet basis, have very sparse representations, thus those signals and images can be recovered from a small amount of measurements with very high accuracy. This paper is concerned with the dimensionality reduction problem based on the compressive assumptions. We propose novel unsupervised and semi-supervised dimensionality reduction algorithms by exploiting sparse data representations. The experiments show that the proposed approaches outperform state-of-the-art dimensionality reduction methods.

[1]  Feiping Nie,et al.  A unified framework for semi-supervised dimensionality reduction , 2008, Pattern Recognit..

[2]  R.G. Baraniuk,et al.  Compressive Sensing [Lecture Notes] , 2007, IEEE Signal Processing Magazine.

[3]  Q. Shi,et al.  Gaussian Process Latent Variable Models for , 2011 .

[4]  N. Cristianini Fisher Discriminant Analysis (Linear Discriminant Analysis) , 2014 .

[5]  Florian Steinke,et al.  Bayesian Inference and Optimal Design in the Sparse Linear Model , 2007, AISTATS.

[6]  M. Rudelson,et al.  Geometric approach to error-correcting codes and reconstruction of signals , 2005, math/0502299.

[7]  Jieping Ye,et al.  Multi-Task Feature Learning Via Efficient l2, 1-Norm Minimization , 2009, UAI.

[8]  Shinichi Nakajima,et al.  Semi-Supervised Local Fisher Discriminant Analysis for Dimensionality Reduction , 2008, PAKDD.

[9]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[10]  Junbin Gao,et al.  Kernel Laplacian Eigenmaps for Visualization of Non-vectorial Data , 2006, Australian Conference on Artificial Intelligence.

[11]  Frank Plastria,et al.  Dimensionality Reduction for Classification , 2008, ADMA.

[12]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[13]  Ingrid Daubechies,et al.  Ten Lectures on Wavelets , 1992 .

[14]  Xin Yang,et al.  Semi-supervised nonlinear dimensionality reduction , 2006, ICML.

[15]  John Langford,et al.  Hash Kernels , 2009, AISTATS.

[16]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[17]  Neil D. Lawrence,et al.  Probabilistic Non-linear Principal Component Analysis with Gaussian Process Latent Variable Models , 2005, J. Mach. Learn. Res..

[18]  张振跃,et al.  Principal Manifolds and Nonlinear Dimensionality Reduction via Tangent Space Alignment , 2004 .

[19]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[20]  Matthias W. Seeger,et al.  Compressed sensing and Bayesian experimental design , 2008, ICML '08.

[21]  Chunhua Shen,et al.  Rapid face recognition using hashing , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Hongyuan Zha,et al.  Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment , 2002, ArXiv.

[23]  Junzhou Huang,et al.  The Benefit of Group Sparsity , 2009 .

[24]  Masashi Sugiyama,et al.  Dimensionality Reduction of Multimodal Labeled Data by Local Fisher Discriminant Analysis , 2007, J. Mach. Learn. Res..

[25]  Richard G. Baraniuk,et al.  Compressive Sensing , 2008, Computer Vision, A Reference Guide.

[26]  Junbin Gao,et al.  Dimensionality Reduction for Classification through Visualisation Using L1SNE , 2010, Australasian Conference on Artificial Intelligence.

[27]  John Langford,et al.  Hash Kernels for Structured Data , 2009, J. Mach. Learn. Res..

[28]  Eric O. Postma,et al.  Dimensionality Reduction: A Comparative Review , 2008 .

[29]  David B. Dunson,et al.  Multi-Task Compressive Sensing , 2007 .

[30]  Yonina C. Eldar,et al.  Compressed Sensing with Coherent and Redundant Dictionaries , 2010, ArXiv.

[31]  Christopher M. Bishop,et al.  GTM: The Generative Topographic Mapping , 1998, Neural Computation.

[32]  David B. Dunson,et al.  Multitask Compressive Sensing , 2009, IEEE Transactions on Signal Processing.

[33]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[34]  Mihailo Stojnic,et al.  Block-length dependent thresholds in block-sparse compressed sensing , 2009, ArXiv.

[35]  G. Obozinski,et al.  High-dimensional union support recovery in multivariate regression , 2008 .

[36]  H. Zha,et al.  Principal manifolds and nonlinear dimensionality reduction via tangent space alignment , 2004, SIAM J. Sci. Comput..

[37]  Jean-Luc Starck,et al.  Compressed sensing with Herschel/PACS data , 2010 .

[38]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[39]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  Daoqiang Zhang,et al.  Semi-Supervised Dimensionality Reduction ∗ , 2007 .

[41]  Michael I. Jordan,et al.  Multi-task feature selection , 2006 .

[42]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[43]  Lawrence Carin,et al.  Bayesian Compressive Sensing , 2008, IEEE Transactions on Signal Processing.

[44]  S. Mallat A wavelet tour of signal processing , 1998 .

[45]  R. Calderbank Compressed Learning : Universal Sparse Dimensionality Reduction and Learning in the Measurement Domain , 2009 .

[46]  Massimiliano Pontil,et al.  Convex multi-task feature learning , 2008, Machine Learning.