Sparsity-Aware Learning and Compressed Sensing: An Overview

This paper is based on a chapter of a new book on Machine Learning, by the first and third author, which is currently under preparation. We provide an overview of the major theoretical advances as well as the main trends in algorithmic developments in the area of sparsity-aware learning and compressed sensing. Both batch processing and online processing techniques are considered. A case study in the context of time-frequency analysis of signals is also presented. Our intent is to update this review from time to time, since this is a very hot research area with a momentum and speed that is sometimes difficult to follow up.

[1]  Heather Bourbeau.  Greed Is Good , 2004 .

[2]  Arvind Ganesh,et al.  Fast algorithms for recovering a corrupted low-rank matrix , 2009, 2009 3rd IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[3]  D. Donoho,et al.  Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA) , 2005 .

[4]  John Wright,et al.  Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization , 2009, NIPS.

[5]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[6]  Junfeng Yang,et al.  A Fast Alternating Direction Method for TVL1-L2 Signal Reconstruction From Partial Fourier Data , 2010, IEEE Journal of Selected Topics in Signal Processing.

[7]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[8]  H. L. Taylor,et al.  Deconvolution with the l 1 norm , 1979 .

[9]  Andreas Peter Burg,et al.  Hardware-efficient random sampling of fourier-sparse signals , 2012, 2012 IEEE International Symposium on Circuits and Systems.

[10]  I. Yamada,et al.  Hybrid Steepest Descent Method for Variational Inequality Problem over the Fixed Point Set of Certain Quasi-nonexpansive Mappings , 2005 .

[11]  Michael P. Friedlander,et al.  Probing the Pareto Frontier for Basis Pursuit Solutions , 2008, SIAM J. Sci. Comput..

[12]  Richard G. Baraniuk,et al.  Random Projections of Smooth Manifolds , 2009, Found. Comput. Math..

[13]  Thierry Blu,et al.  Extrapolation and Interpolation) , 2022 .

[14]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[15]  I. Yamada,et al.  The Adaptive Projected Subgradient Method over the Fixed Point Set of Strongly Attracting Nonexpansive Mappings , 2006 .

[16]  Nathan Srebro,et al.  Concentration-Based Guarantees for Low-Rank Matrix Reconstruction , 2011, COLT.

[17]  Gongguo Tang,et al.  Performance Analysis of Sparse Recovery Based on Constrained Minimal Singular Values , 2010, IEEE Transactions on Signal Processing.

[18]  Justin K. Romberg,et al.  Beyond Nyquist: Efficient Sampling of Sparse Bandlimited Signals , 2009, IEEE Transactions on Information Theory.

[19]  Juha Karhunen,et al.  Generalizations of principal component analysis, optimization problems, and neural networks , 1995, Neural Networks.

[20]  Mike E. Davies,et al.  Sampling Theorems for Signals From the Union of Finite-Dimensional Linear Subspaces , 2009, IEEE Transactions on Information Theory.

[21]  Michael Elad,et al.  The Cosparse Analysis Model and Algorithms , 2011, ArXiv.

[22]  Richard G. Baraniuk,et al.  Asymptotic Analysis of Complex LASSO via Complex Approximate Message Passing (CAMP) , 2011, IEEE Transactions on Information Theory.

[23]  Kostas Berberidis,et al.  Efficient decision feedback equalization for sparse wireless channels , 2003, IEEE Trans. Wirel. Commun..

[24]  M. Salman Asif,et al.  Dynamic Updating for ` 1 Minimization , 2009 .

[25]  Bhaskar D. Rao,et al.  Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm , 1997, IEEE Trans. Signal Process..

[26]  J. Claerbout,et al.  Robust Modeling With Erratic Data , 1973 .

[27]  H. Zou,et al.  One-step Sparse Estimates in Nonconcave Penalized Likelihood Models. , 2008, Annals of statistics.

[28]  Volkan Cevher,et al.  Low-Dimensional Models for Dimensionality Reduction and Signal Recovery: A Geometric Perspective , 2010, Proceedings of the IEEE.

[29]  Sergios Theodoridis,et al.  Generalized thresholding sparsity-aware algorithm for low complexity online learning , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[30]  I. Johnstone,et al.  Wavelet Shrinkage: Asymptopia? , 1995 .

[31]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[32]  Robert D. Nowak,et al.  An EM algorithm for wavelet-based image restoration , 2003, IEEE Trans. Image Process..

[33]  Sergey Bakin,et al.  Adaptive regression and model selection in data mining problems , 1999 .

[34]  Michael Elad,et al.  From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images , 2009, SIAM Rev..

[35]  Wotao Yin,et al.  Bregman Iterative Algorithms for (cid:2) 1 -Minimization with Applications to Compressed Sensing ∗ , 2008 .

[36]  Lloyd R. Welch,et al.  Lower bounds on the maximum cross correlation of signals (Corresp.) , 1974, IEEE Trans. Inf. Theory.

[37]  M. Vetterli,et al.  Sparse Sampling of Signal Innovations , 2008, IEEE Signal Processing Magazine.

[38]  D. Donoho,et al.  Counting faces of randomly-projected polytopes when the projection radically lowers dimension , 2006, math/0607364.

[39]  F. Santosa,et al.  Linear inversion of ban limit reflection seismograms , 1986 .

[40]  Thierry Blu,et al.  Sampling signals with finite rate of innovation , 2002, IEEE Trans. Signal Process..

[41]  Michael Elad,et al.  Dictionaries for Sparse Representation Modeling , 2010, Proceedings of the IEEE.

[42]  M. R. Osborne,et al.  A new approach to variable selection in least squares problems , 2000 .

[43]  R. Calderbank,et al.  Compressed Learning : Universal Sparse Dimensionality Reduction and Learning in the Measurement Domain , 2009 .

[44]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[45]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[46]  B. Recht,et al.  Tensor completion and low-n-rank tensor recovery via convex optimization , 2011 .

[47]  J. Hiriart-Urruty,et al.  Fundamentals of Convex Analysis , 2004 .

[48]  Elias Aboutanios,et al.  Time-frequency and advanced frequency estimation techniques for the investigation of bat echolocation calls. , 2010, The Journal of the Acoustical Society of America.

[49]  Mark D. Plumbley.  Geometry and homotopy for l 1 sparse representations , 2005 .

[50]  B. Logan,et al.  Signal recovery and the large sieve , 1992 .

[51]  J. Tropp,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, Commun. ACM.

[52]  Sergios Theodoridis,et al.  Online Sparse System Identification and Signal Reconstruction Using Projections Onto Weighted $\ell_{1}$ Balls , 2010, IEEE Transactions on Signal Processing.

[53]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.

[54]  Yonina C. Eldar,et al.  Structured Compressed Sensing: From Theory to Applications , 2011, IEEE Transactions on Signal Processing.

[55]  Pascal Frossard,et al.  Dictionary learning: What is the right representation for my signal? , 2011 .

[56]  Mohamed-Jalal Fadili,et al.  The Undecimated Wavelet Decomposition and its Reconstruction , 2007, IEEE Transactions on Image Processing.

[57]  M. Wakin.  Manifold-Based Signal Recovery and Parameter Estimation from Compressive Measurements , 2010, 1002.1247.

[58]  Pablo A. Parrilo,et al.  Rank-Sparsity Incoherence for Matrix Decomposition , 2009, SIAM J. Optim..

[59]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[60]  S. Kirolos,et al.  Analog-to-Information Conversion via Random Demodulation , 2006, 2006 IEEE Dallas/CAS Workshop on Design, Applications, Integration and Software.

[61]  Minh N. Do,et al.  A Theory for Sampling Signals from a Union of Subspaces , 2022 .

[62]  Aswin C. Sankaranarayanan,et al.  SpaRCS: Recovering low-rank and sparse matrices from compressive measurements , 2011, NIPS.

[63]  Monisha Ghosh.  Blind decision feedback equalization for terrestrial television receivers , 1998 .

[64]  P. Vaidyanathan,et al.  Periodically nonuniform sampling of bandpass signals , 1998 .

[65]  Georgios B. Giannakis,et al.  Compressed Sensing for Wideband Cognitive Radios , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[66]  K. Slavakis,et al.  Reduced complexity online sparse signal reconstruction using projections onto weighted ℓ1 balls , 2011, 2011 17th International Conference on Digital Signal Processing (DSP).

[67]  Justin K. Romberg,et al.  On the LASSO and Dantzig selector equivalence , 2010, 2010 44th Annual Conference on Information Sciences and Systems (CISS).

[68]  Taghi M. Khoshgoftaar,et al.  A Survey of Collaborative Filtering Techniques , 2009, Adv. Artif. Intell..

[69]  Michael Elad,et al.  L1-L2 Optimization in Signal and Image Processing , 2010, IEEE Signal Processing Magazine.

[70]  Lieven Vandenberghe,et al.  Interior-Point Method for Nuclear Norm Approximation with Application to System Identification , 2009, SIAM J. Matrix Anal. Appl..

[71]  I. Daubechies,et al.  Iteratively reweighted least squares minimization for sparse recovery , 2008, 0807.0575.

[72]  George B. Dantzig,et al.  Linear programming and extensions , 1965 .

[73]  Yonina C. Eldar,et al.  Compressed Sensing with Coherent and Redundant Dictionaries , 2010, ArXiv.

[74]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[75]  Johan A. K. Suykens,et al.  Tensor Versus Matrix Completion: A Comparison With Application to Spectral Data , 2011, IEEE Signal Processing Letters.

[76]  Michael J. Black,et al.  A Framework for Robust Subspace Learning , 2003, International Journal of Computer Vision.

[77]  Pascal Frossard,et al.  Dictionary Learning , 2011, IEEE Signal Processing Magazine.

[78]  Sanjoy Dasgupta,et al.  Experiments with Random Projection , 2000, UAI.

[79]  Simon Foucart,et al.  Hard Thresholding Pursuit: An Algorithm for Compressive Sensing , 2011, SIAM J. Numer. Anal..

[80]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[81]  Wenjiang J. Fu,et al.  Asymptotics for lasso-type estimators , 2000 .

[82]  P. Tseng,et al.  On the convergence of the coordinate descent method for convex differentiable minimization , 1992 .

[83]  Yonina C. Eldar,et al.  Sub-Nyquist Sampling of Short Pulses , 2010, IEEE Transactions on Signal Processing.

[84]  Arian Maleki,et al.  Optimally Tuned Iterative Reconstruction Algorithms for Compressed Sensing , 2009, IEEE Journal of Selected Topics in Signal Processing.

[85]  Nesa L'abbe Wu,et al.  Linear programming and extensions , 1981 .

[86]  J. Tropp,et al.  Two proposals for robust PCA using semidefinite programming , 2010, 1012.1086.

[87]  Justin K. Romberg,et al.  Dynamic Updating for $\ell_{1}$ Minimization , 2009, IEEE Journal of Selected Topics in Signal Processing.

[88]  Rama Chellappa,et al.  Domain Adaptive Dictionary Learning , 2012, ECCV.

[89]  Jian-Feng Cai,et al.  Split Bregman Methods and Frame Based Image Restoration , 2009, Multiscale Model. Simul..

[90]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[91]  I. Daubechies,et al.  PAINLESS NONORTHOGONAL EXPANSIONS , 1986 .

[92]  G. Papavassilopoulos,et al.  On the rank minimization problem over a positive semidefinite linear matrix inequality , 1997, IEEE Trans. Autom. Control..

[93]  Vahid Tarokh,et al.  An Adaptive Greedy Algorithm With Application to Nonlinear Communications , 2010, IEEE Transactions on Signal Processing.

[94]  Rodney G. Vaughan,et al.  The theory of bandpass sampling , 1991, IEEE Trans. Signal Process..

[95]  Tom Goldstein,et al.  The Split Bregman Method for L1-Regularized Problems , 2009, SIAM J. Imaging Sci..

[96]  Yonina C. Eldar,et al.  Xampling: Analog Data Compression , 2010, 2010 Data Compression Conference.

[97]  I. Johnstone,et al.  Maximum entropy reconstruction of complex (phase-sensitive) spectra , 1990 .

[98]  Michael Elad,et al.  Analysis versus synthesis in signal priors , 2006, 2006 14th European Signal Processing Conference.

[99]  David Gross,et al.  Recovering Low-Rank Matrices From Few Coefficients in Any Basis , 2009, IEEE Transactions on Information Theory.

[100]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[101]  Stephen J. Wright,et al.  Sparse reconstruction by separable approximation , 2009, IEEE Trans. Signal Process..

[102]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[103]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[104]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[105]  Ali H. Sayed,et al.  Fundamentals Of Adaptive Filtering , 2003 .

[106]  I. Johnstone,et al.  Ideal spatial adaptation by wavelet shrinkage , 1994 .

[107]  Jacob Benesty,et al.  Advances in Network and Acoustic Echo Cancellation , 2001 .

[108]  S. Mendelson,et al.  Uniform Uncertainty Principle for Bernoulli and Subgaussian Ensembles , 2006, math/0608665.

[109]  Deanna Needell,et al.  Stable Image Reconstruction Using Total Variation Minimization , 2012, SIAM J. Imaging Sci..

[110]  Mike E. Davies,et al.  Iterative Hard Thresholding for Compressed Sensing , 2008, ArXiv.

[111]  Michael Elad,et al.  Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing , 2010 .

[112]  Ingrid Daubechies,et al.  Time-frequency localization operators: A geometric phase space approach , 1988, IEEE Trans. Inf. Theory.

[113]  Xiaodong Li,et al.  Stable Principal Component Pursuit , 2010, 2010 IEEE International Symposium on Information Theory.

[114]  Brian M. Sadler,et al.  Mixed-signal parallel compressed sensing and reception for cognitive radio , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[115]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[116]  Ronald A. DeVore,et al.  Some remarks on greedy algorithms , 1996, Adv. Comput. Math..

[117]  Xiaoming Yuan,et al.  Matrix completion via an alternating direction method , 2012 .

[118]  Yonina C. Eldar,et al.  Sub-Nyquist Sampling , 2011, IEEE Signal Processing Magazine.

[119]  Vladimir Stojanovic,et al.  Design and Analysis of a Hardware-Efficient Compressed Sensing Architecture for Data Compression in Wireless Sensors , 2012, IEEE Journal of Solid-State Circuits.

[120]  Aníbal R. Figueiras-Vidal,et al.  Adaptive Combination of Proportionate Filters for Sparse Echo Cancellation , 2009, IEEE Transactions on Audio, Speech, and Language Processing.

[121]  Volkan Cevher,et al.  Sparse Signal Recovery and Acquisition with Graphical Models , 2010, IEEE Signal Processing Magazine.

[122]  Tong Zhang,et al.  Sparse Recovery With Orthogonal Matching Pursuit Under RIP , 2010, IEEE Transactions on Information Theory.

[123]  Jun Zhang,et al.  On Recovery of Sparse Signals via ℓ1 Minimization , 2008, ArXiv.

[124]  S. Yun,et al.  An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems , 2009 .

[125]  Bruno A. Olshausen,et al.  Learning Horizontal Connections in a Sparse Coding Model of Natural Images , 2007, NIPS.

[126]  D. Donoho,et al.  Neighborliness of randomly projected simplices in high dimensions. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[127]  Rajeev Motwani,et al.  Randomized Algorithms , 1995, SIGA.

[128]  Michael Elad,et al.  A generalized uncertainty principle and sparse representation in pairs of bases , 2002, IEEE Trans. Inf. Theory.

[129]  L. Greenstein,et al.  Tap-selectable decision-feedback equalization , 1997, IEEE Trans. Commun..

[130]  J. Kovacevic,et al.  Life Beyond Bases: The Advent of Frames (Part II) , 2007, IEEE Signal Processing Magazine.

[131]  Mário A. T. Figueiredo,et al.  Signal restoration with overcomplete wavelet transforms: comparison of analysis and synthesis priors , 2009, Optical Engineering + Applications.

[132]  R. Rockafellar.  Monotone Operators and the Proximal Point Algorithm , 1976 .

[133]  Volkan Cevher,et al.  Model-Based Compressive Sensing , 2008, IEEE Transactions on Information Theory.

[134]  Xiaoming Yuan,et al.  Sparse and low-rank matrix decomposition via alternating direction method , 2013 .

[135]  Anestis Antoniadis,et al.  Wavelet methods in statistics: Some recent developments and their applications , 2007, 0712.0283.

[136]  Stephen P. Boyd,et al.  A rank minimization heuristic with application to minimum order system approximation , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[137]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[138]  R. Tibshirani,et al.  A note on the group lasso and a sparse group lasso , 2010, 1001.0736.

[139]  Robert D. Nowak,et al.  Toeplitz Compressed Sensing Matrices With Applications to Sparse Channel Estimation , 2010, IEEE Transactions on Information Theory.

[140]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.

[141]  R. DeVore,et al.  Compressed sensing and best k-term approximation , 2008 .

[142]  Stephen J. Wright,et al.  Sparse Reconstruction by Separable Approximation , 2008, IEEE Transactions on Signal Processing.

[143]  Dmitry M. Malioutov,et al.  Homotopy continuation for sparse signal representation , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[144]  Yonina C. Eldar,et al.  Block-Sparse Signals: Uncertainty Relations and Efficient Recovery , 2009, IEEE Transactions on Signal Processing.

[145]  Michael B. Wakin,et al.  Analysis of Orthogonal Matching Pursuit Using the Restricted Isometry Property , 2009, IEEE Transactions on Information Theory.

[146]  Benjamin Recht,et al.  A Simpler Approach to Matrix Completion , 2009, J. Mach. Learn. Res..

[147]  P. Bickel,et al.  SIMULTANEOUS ANALYSIS OF LASSO AND DANTZIG SELECTOR , 2008, 0801.1095.

[148]  David L. Donoho,et al.  Counting the Faces of Randomly-Projected Hypercubes and Orthants, with Applications , 2008, Discret. Comput. Geom..

[149]  Sergios Theodoridis,et al.  Adaptive Learning in a World of Projections , 2011, IEEE Signal Processing Magazine.

[150]  T. Strohmer,et al.  Gabor Analysis and Algorithms: Theory and Applications , 1997 .

[151]  Yehoshua Y. Zeevi,et al.  Frame analysis of the discrete Gabor-scheme , 1994, IEEE Trans. Signal Process..

[152]  Mike E. Davies,et al.  Parametric Dictionary Design for Sparse Coding , 2009, IEEE Transactions on Signal Processing.

[153]  Jun Zhang,et al.  On Recovery of Sparse Signals Via $\ell _{1}$ Minimization , 2008, IEEE Transactions on Information Theory.

[154]  Alan L. Yuille,et al.  Robust principal component analysis by self-organizing rules based on statistical physics approach , 1995, IEEE Trans. Neural Networks.

[155]  Xiaodong Li,et al.  Dense error correction for low-rank matrices via Principal Component Pursuit , 2010, 2010 IEEE International Symposium on Information Theory.

[156]  Deguang Han,et al.  Frames, bases, and group representations , 2000 .

[157]  Boualem Boashash,et al.  Time Frequency Analysis , 2003 .

[158]  Y. Nesterov.  A method for solving the convex programming problem with convergence rate O(1/k^2) , 1983 .

[159]  Mike E. Davies,et al.  Normalized Iterative Hard Thresholding: Guaranteed Stability and Performance , 2010, IEEE Journal of Selected Topics in Signal Processing.

[160]  Georgios B. Giannakis,et al.  Online Adaptive Estimation of Sparse Signals: Where RLS Meets the $\ell_1$ -Norm , 2010, IEEE Transactions on Signal Processing.

[161]  Yoram Singer,et al.  Efficient projections onto the l1-ball for learning in high dimensions , 2008, ICML '08.

[162]  D. Takhar,et al.  A compressed sensing camera : New theory and an implementation using digital micromirrors , 2006 .

[163]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[164]  Adrian Stern,et al.  Compressed Imaging With a Separable Sensing Operator , 2009, IEEE Signal Processing Letters.

[165]  H. Zou.  The Adaptive Lasso and Its Oracle Properties , 2006 .

[166]  Edward J. Wegman,et al.  Statistical Signal Processing , 1985 .

[167]  Michael Elad,et al.  Coordinate and subspace optimization methods for linear least squares with non-quadratic regularization , 2007 .

[168]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[169]  Massimiliano Pontil,et al.  Multi-Task Feature Learning , 2006, NIPS.

[170]  Stephen P. Boyd,et al.  An Interior-Point Method for Large-Scale $\ell_1$-Regularized Least Squares , 2007, IEEE Journal of Selected Topics in Signal Processing.

[171]  Holger Rauhut,et al.  Multichannel-compressive estimation of doubly selective channels in MIMO-OFDM systems: Exploiting and enhancing joint sparsity , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[172]  Yonina C. Eldar,et al.  C-HiLasso: A Collaborative Hierarchical Sparse Modeling Framework , 2010, IEEE Transactions on Signal Processing.

[173]  Emmanuel J. Candès,et al.  The curvelet transform for image denoising , 2002, IEEE Trans. Image Process..

[174]  Richard Baraniuk,et al.  Recovery of Clustered Sparse Signals from Compressive Measurements , 2009 .

[175]  Justin Romberg,et al.  Practical Signal Recovery from Random Projections , 2005 .

[176]  Avrim Blum,et al.  Random Projection, Margins, Kernels, and Feature-Selection , 2005, SLSFS.

[177]  Nick G. Kingsbury,et al.  Overcomplete image coding using iterative projection-based noise shaping , 2002, Proceedings. International Conference on Image Processing.

[178]  John Wright,et al.  Compressive principal component pursuit , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[179]  R. Duffin,et al.  A class of nonharmonic Fourier series , 1952 .

[180]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[181]  Yoram Bresler,et al.  Perfect reconstruction formulas and bounds on aliasing error in sub-nyquist nonuniform sampling of multiband signals , 2000, IEEE Trans. Inf. Theory.

[182]  Vladimir N. Temlyakov,et al.  Nonlinear Methods of Approximation , 2003, Found. Comput. Math..

[183]  Charles L. Byrne,et al.  Applied Iterative Methods , 2007 .

[184]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[185]  David M. Young,et al.  Applied Iterative Methods , 2004 .

[186]  John Langford,et al.  Sparse Online Learning via Truncated Gradient , 2008, NIPS.

[187]  Brian D. O. Anderson,et al.  Wireless sensor network localization techniques , 2007, Comput. Networks.

[188]  Michael I. Jordan,et al.  Multi-task feature selection , 2006 .

[189]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[190]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[191]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[192]  Dimitris Achlioptas,et al.  Database-friendly random projections , 2001, PODS.

[193]  Bastian Goldlücke,et al.  Variational Analysis , 2014, Computer Vision, A Reference Guide.

[194]  A. Montanari,et al.  On positioning via distributed matrix completion , 2010, 2010 IEEE Sensor Array and Multichannel Signal Processing Workshop.

[195]  Yonina C. Eldar,et al.  Xampling: Signal Acquisition and Processing in Union of Subspaces , 2009, IEEE Transactions on Signal Processing.

[196]  P. Jansson.  Deconvolution : with applications in spectroscopy , 1984 .

[197]  A. TroppJ..  Greed is good , 2006 .

[198]  I. Yamada,et al.  NON-STRICTLY CONVEX MINIMIZATION OVER THE FIXED POINT SET OF AN ASYMPTOTICALLY SHRINKING NONEXPANSIVE MAPPING , 2002 .

[199]  Yue M. Lu,et al.  Sampling Signals from a Union of Subspaces , 2008, IEEE Signal Processing Magazine.

[200]  Rémi Gribonval,et al.  Sparse representations in unions of bases , 2003, IEEE Trans. Inf. Theory.

[201]  Constantine Caramanis,et al.  Robust PCA via Outlier Pursuit , 2010, IEEE Transactions on Information Theory.

[202]  Babak Hassibi,et al.  Recovering Sparse Signals Using Sparse Measurement Matrices in Compressed DNA Microarrays , 2008, IEEE Journal of Selected Topics in Signal Processing.

[203]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .

[204]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[205]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[206]  Paulo Sergio Ramirez,et al.  Fundamentals of Adaptive Filtering , 2002 .

[207]  David L. Donoho,et al.  Precise Undersampling Theorems , 2010, Proceedings of the IEEE.

[208]  G. Sapiro,et al.  A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography. , 2013, Journal of structural biology.

[209]  Emmanuel J. Candès,et al.  NESTA: A Fast and Accurate First-Order Method for Sparse Recovery , 2009, SIAM J. Imaging Sci..

[210]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[211]  Ronald R. Coifman,et al.  Entropy-based algorithms for best basis selection , 1992, IEEE Trans. Inf. Theory.

[212]  Stephen P. Boyd,et al.  Rank minimization and applications in system theory , 2004, Proceedings of the 2004 American Control Conference.

[213]  Stéphane Mallat,et al.  A Wavelet Tour of Signal Processing - The Sparse Way, 3rd Edition , 2008 .

[214]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[215]  Mia Hubert,et al.  ROBPCA: A New Approach to Robust Principal Component Analysis , 2005, Technometrics.

[216]  Mia Hubert,et al.  Robust PCA and classification in biosciences , 2004, Bioinform..

[217]  Jelena Kovačević,et al.  Life Beyond Bases: The Advent of Frames , 2006 .

[218]  M. Unser.  Sampling-50 years after Shannon , 2000, Proceedings of the IEEE.

[219]  M. Rudelson,et al.  On sparse reconstruction from Fourier and Gaussian measurements , 2008 .

[220]  Mike Brookes,et al.  Adaptive algorithms for sparse echo cancellation , 2006, Signal Process..

[221]  Olgica Milenkovic,et al.  Subspace Pursuit for Compressive Sensing Signal Reconstruction , 2008, IEEE Transactions on Information Theory.

[222]  Patrick Flandrin,et al.  Time-Frequency/Time-Scale Analysis , 1998 .

[223]  David Suter,et al.  Recovering the missing components in a large noisy low-rank matrix: application to SFM , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[224]  Bhaskar D. Rao,et al.  Matching pursuit based decision-feedback equalizers , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[225]  Yoram Bresler,et al.  ADMiRA: Atomic Decomposition for Minimum Rank Approximation , 2009, IEEE Transactions on Information Theory.

[226]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[227]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[228]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[229]  Robert D. Nowak,et al.  Compressed Channel Sensing: A New Approach to Estimating Sparse Multipath Channels , 2010, Proceedings of the IEEE.

[230]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[231]  Franz Hlawatsch,et al.  Group sparsity methods for compressive channel estimation in doubly dispersive multicarrier systems , 2010, 2010 IEEE 11th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC).

[232]  Richard G. Baraniuk,et al.  Kronecker Compressive Sensing , 2012, IEEE Transactions on Image Processing.

[233]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[234]  Yinyu Ye,et al.  Semidefinite programming based algorithms for sensor network localization , 2006, TOSN.