Oblique Pursuits for Compressed Sensing

Compressed sensing is a new data acquisition paradigm enabling universal, simple, and reduced-cost acquisition, by exploiting a sparse signal model. Most notably, recovery of the signal by computationally efficient algorithms is guaranteed for certain randomized acquisition systems. However, there is a discrepancy between the theoretical guarantees and practical applications. In applications, including Fourier imaging in various modalities, the measurements are acquired by inner products with vectors selected randomly (sampled) from a frame. Currently available guarantees are derived using the so-called restricted isometry property (RIP), which has only been shown to hold under ideal assumptions. For example, the sampling from the frame needs to be independent and identically distributed with the uniform distribution, and the frame must be tight. In practice though, one or more of the ideal assumptions are typically violated and none of the RIP-based guarantees applies. Motivated by this discrepancy, we propose two related changes in the existing framework: 1) a generalized RIP called the restricted biorthogonality property (RBOP); and 2) correspondingly modified versions of existing greedy pursuit algorithms, which we call oblique pursuits. Oblique pursuits are guaranteed using the RBOP without requiring ideal assumptions; hence, the guarantees apply to practical acquisition schemes. Numerical results show that oblique pursuits also perform competitively with, or sometimes better than their conventional counterparts.

[1]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[2]  Richard Kueng,et al.  RIPless compressed sensing from anisotropic measurements , 2012, ArXiv.

[3]  Ilse C. F. Ipsen,et al.  The Angle Between Complementary Subspaces , 1995 .

[4]  Volkan Cevher,et al.  Fast hard thresholding with Nesterov"s gradient method , 2010, NIPS 2010.

[5]  Tong Zhang,et al.  Sparse Recovery With Orthogonal Matching Pursuit Under RIP , 2010, IEEE Transactions on Information Theory.

[6]  Y. Bresler Spectrum-blind sampling and compressive sensing for continuous-index signals , 2008, 2008 Information Theory and Applications Workshop.

[7]  S. Foucart Sparse Recovery Algorithms: Sufficient Conditions in Terms of RestrictedIsometry Constants , 2012 .

[8]  Yoram Bresler,et al.  Fast Tomographic Reconstruction via Rotation-Based Hierarchical Backprojection , 2007, SIAM J. Appl. Math..

[9]  O. Christensen An introduction to frames and Riesz bases , 2002 .

[10]  Ronald L. Smith Some interlacing properties of the Schur complement of a Hermitian matrix , 1992 .

[11]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[12]  E. Candès,et al.  Sparsity and incoherence in compressive sampling , 2006, math/0611957.

[13]  Yoram Bresler,et al.  O(N2log2N) filtered backprojection reconstruction algorithm for tomography , 2000, IEEE Trans. Image Process..

[14]  Shuheng Zhou,et al.  25th Annual Conference on Learning Theory Reconstruction from Anisotropic Random Measurements , 2022 .

[15]  Jeffrey A. Fessler,et al.  Nonuniform fast Fourier transforms using min-max interpolation , 2003, IEEE Trans. Signal Process..

[16]  Yoram Bresler,et al.  Subspace Methods for Joint Sparse Recovery , 2010, IEEE Transactions on Information Theory.

[17]  H. Rauhut Compressive Sensing and Structured Random Matrices , 2009 .

[18]  R. Dudley The Sizes of Compact Subsets of Hilbert Space and Continuity of Gaussian Processes , 1967 .

[19]  M. Talagrand,et al.  Probability in Banach Spaces: Isoperimetry and Processes , 1991 .

[20]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .

[21]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[22]  Mike E. Davies,et al.  Iterative Hard Thresholding for Compressed Sensing , 2008, ArXiv.

[23]  R. Baraniuk,et al.  Compressive Radar Imaging , 2007, 2007 IEEE Radar Conference.

[24]  Olgica Milenkovic,et al.  Subspace Pursuit for Compressive Sensing Signal Reconstruction , 2008, IEEE Transactions on Information Theory.

[25]  Jong Chul Ye,et al.  Compressive Diffuse Optical Tomography: Noniterative Exact Reconstruction Using Joint Sparsity , 2011, IEEE Transactions on Medical Imaging.

[26]  Yonina C. Eldar,et al.  From Theory to Practice: Sub-Nyquist Sampling of Sparse Wideband Analog Signals , 2009, IEEE Journal of Selected Topics in Signal Processing.

[27]  Martin J. Wainwright,et al.  A unified framework for high-dimensional analysis of $M$-estimators with decomposable regularizers , 2009, NIPS.

[28]  Yoram Bresler,et al.  Further results on spectrum blind sampling of 2D signals , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[29]  Pierre Moulin,et al.  A self-referencing level-set method for image reconstruction from sparse Fourier samples , 2001, Proceedings IEEE Workshop on Variational and Level Set Methods in Computer Vision.

[30]  Pierre Vandergheynst,et al.  Compressed Sensing and Redundant Dictionaries , 2007, IEEE Transactions on Information Theory.

[31]  Martin J. Wainwright,et al.  Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _{1}$ -Constrained Quadratic Programming (Lasso) , 2009, IEEE Transactions on Information Theory.

[32]  Rachel Ward,et al.  Stable and Robust Sampling Strategies for Compressive Imaging , 2012, IEEE Transactions on Image Processing.

[33]  Holger Rauhut,et al.  Compressive Sensing with structured random matrices , 2012 .

[34]  Rachel Ward,et al.  New and Improved Johnson-Lindenstrauss Embeddings via the Restricted Isometry Property , 2010, SIAM J. Math. Anal..

[35]  Yoram Bresler,et al.  Learning Sparsifying Transforms , 2013, IEEE Transactions on Signal Processing.

[36]  Ping Feng,et al.  Spectrum-blind minimum-rate sampling and reconstruction of multiband signals , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[37]  M. Rudelson,et al.  On sparse reconstruction from Fourier and Gaussian measurements , 2008 .

[38]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[39]  Yoram Bresler,et al.  MR Image Reconstruction From Highly Undersampled k-Space Data by Dictionary Learning , 2011, IEEE Transactions on Medical Imaging.

[40]  Ping Feng,et al.  Universal Minimum-Rate Sampling and Spectrum-Blind Reconstruction for Multiband Signals , 1998 .

[41]  Yonina C. Eldar Sampling with Arbitrary Sampling and Reconstruction Spaces and Oblique Dual Frame Vectors , 2003 .

[42]  Holger Rauhut Stability Results for Random Sampling of Sparse Trigonometric Polynomials , 2008, IEEE Transactions on Information Theory.

[43]  S. Geer,et al.  On the conditions used to prove oracle results for the Lasso , 2009, 0910.0722.

[44]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[45]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[46]  Jean-Luc Starck,et al.  Compressed Sensing in Astronomy , 2008, IEEE Journal of Selected Topics in Signal Processing.

[47]  Massimo Fornasier,et al.  Compressive Sensing and Structured Random Matrices , 2010 .

[48]  Pierre Vandergheynst,et al.  Dictionary Preconditioning for Greedy Algorithms , 2008, IEEE Transactions on Signal Processing.

[49]  Volkan Cevher,et al.  Model-Based Compressive Sensing , 2008, IEEE Transactions on Information Theory.

[50]  Inderjit S. Dhillon,et al.  Orthogonal Matching Pursuit with Replacement , 2011, NIPS.

[51]  Thomas Strohmer,et al.  High-Resolution Radar via Compressed Sensing , 2008, IEEE Transactions on Signal Processing.

[52]  Y. Bresler,et al.  Spectrum-blind minimum-rate sampling and reconstruction of 2-D multiband signals , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[53]  Yonina C. Eldar,et al.  Blind Multiband Signal Reconstruction: Compressed Sensing for Analog Signals , 2007, IEEE Transactions on Signal Processing.

[54]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[55]  Frédéric Lesage,et al.  The Application of Compressed Sensing for Photo-Acoustic Tomography , 2009, IEEE Transactions on Medical Imaging.

[56]  Stéphane Mallat,et al.  A Wavelet Tour of Signal Processing - The Sparse Way, 3rd Edition , 2008 .

[57]  Anna Melnikov,et al.  Studies in Memory of Issai Schur , 2012 .

[58]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[59]  P. Bickel,et al.  SIMULTANEOUS ANALYSIS OF LASSO AND DANTZIG SELECTOR , 2008, 0801.1095.

[60]  Deanna Needell,et al.  Signal Space CoSaMP for Sparse Recovery With Redundant Dictionaries , 2012, IEEE Transactions on Information Theory.

[61]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[62]  Simon Foucart,et al.  RECOVERING JOINTLY SPARSE VECTORS VIA HARD THRESHOLDING PURSUIT , 2011 .

[63]  Yonina C. Eldar,et al.  Xampling: Signal Acquisition and Processing in Union of Subspaces , 2009, IEEE Transactions on Signal Processing.

[64]  Emmanuel J. Candès,et al.  A Probabilistic and RIPless Theory of Compressed Sensing , 2010, IEEE Transactions on Information Theory.

[65]  Yonina C. Eldar,et al.  Compressed Sensing with Coherent and Redundant Dictionaries , 2010, ArXiv.

[66]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[67]  Yonina C. Eldar Compressed Sensing of Analog Signals in Shift-Invariant Spaces , 2008, IEEE Transactions on Signal Processing.

[68]  E. Candès The restricted isometry property and its implications for compressed sensing , 2008 .

[69]  Emre Ertin,et al.  Sparsity and Compressed Sensing in Radar Imaging , 2010, Proceedings of the IEEE.

[70]  V. Paulsen Completely Bounded Maps and Operator Algebras: Completely Bounded Multilinear Maps and the Haagerup Tensor Norm , 2003 .

[71]  Justin K. Romberg,et al.  Beyond Nyquist: Efficient Sampling of Sparse Bandlimited Signals , 2009, IEEE Transactions on Information Theory.

[72]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[73]  David L. Donoho,et al.  Precise Undersampling Theorems , 2010, Proceedings of the IEEE.

[74]  Emmanuel J. Candès,et al.  NESTA: A Fast and Accurate First-Order Method for Sparse Recovery , 2009, SIAM J. Imaging Sci..

[75]  P. Vaidyanathan Multirate Systems And Filter Banks , 1992 .

[76]  Richard E. Blahut,et al.  Theory of Remote Image Formation , 2004 .

[77]  Michael B. Wakin,et al.  Analysis of Orthogonal Matching Pursuit Using the Restricted Isometry Property , 2009, IEEE Transactions on Information Theory.