Compressive Sensing and Structured Random Matrices

These notes give a mathematical introduction to compressive sensing focusing on recovery using `1-minimization and structured random matrices. An emphasis is put on techniques for proving probabilistic estimates for condition numbers of structured random matrices. Estimates of this type are key to providing conditions that ensure exact or approximate recovery of sparse vectors using `1-minimization.

[1]  A. Khintchine.  Über dyadische Brüche , 1923 .

[2]  H. Chernoff.  A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations , 1952 .

[3]  W. Rudin,et al.  Fourier Analysis on Groups. , 1965 .

[4]  G. Bennett.  Probability Inequalities for the Sum of Independent Random Variables , 1962 .

[5]  W. Hoeffding.  Probability Inequalities for sums of Bounded Random Variables , 1963 .

[6]  J. Tukey,et al.  An algorithm for the machine calculation of complex Fourier series , 1965 .

[7]  R. Dudley.  The Sizes of Compact Subsets of Hilbert Space and Continuity of Gaussian Processes , 1967 .

[8]  Sheldon M. Ross,et al.  Introduction to probability models , 1975 .

[9]  B. S. Kašin,et al.  DIAMETERS OF SOME FINITE-DIMENSIONAL SETS AND CLASSES OF SMOOTH FUNCTIONS , 1977 .

[10]  J. Kuelbs.  Probability on Banach spaces , 1978 .

[11]  B. Simon.  Trace ideals and their applications , 1979 .

[12]  William O. Alltop,et al.  Complex sequences with low periodic correlations , 1980 .

[13]  G. Pisier.  Remarques sur un résultat non publié de B. Maurey , 1981 .

[14]  U. Haagerup.  The best constants in the Khintchine inequality , 1981 .

[15]  K. Alexander,et al.  Probability Inequalities for Empirical Processes and a Law of the Iterated Logarithm , 1984 .

[16]  B. Carl.  Inequalities of Bernstein-Jackson-type and the degree of compactness of operators in Banach spaces , 1985 .

[17]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[18]  P. Massart.  Rates of convergence in the central limit theorem for empirical processes , 1986 .

[19]  M. Taqqu,et al.  Decoupling Inequalities for Multilinear Forms in Independent Symmetric Random Variables , 1986 .

[20]  J. Bourgain,et al.  Invertibility of ‘large’ submatrices with applications to the geometry of Banach spaces and harmonic analysis , 1987 .

[21]  M. Talagrand.  Isoperimetry and Integrability of the Sum of Independent Banach-Space Valued Random Variables , 1989 .

[22]  J. Bourgain.  Bounded orthogonal systems and the Λ(p)-set problem , 1989 .

[23]  A. Pinkus.  On L1-Approximation , 1989 .

[24]  B. Bollobás.  THE VOLUME OF CONVEX BODIES AND BANACH SPACE GEOMETRY (Cambridge Tracts in Mathematics 94) , 1991 .

[25]  Charles R. Johnson,et al.  Topics in Matrix Analysis , 1991 .

[26]  Ingrid Daubechies,et al.  Ten Lectures on Wavelets , 1992 .

[27]  G. Pisier.  Non-commutative vector valued Lp-spaces and completely p-summing maps , 1993, math/9306206.

[28]  V. Hardman.  Author Information , 2021, Disability and Health Journal.

[29]  G. Peskir.  Best constants in Kahane-Khintchine inequalities for complex Steinhaus functions , 1995 .

[30]  G. Folland.  A course in abstract harmonic analysis , 1995 .

[31]  A. Shiryaev,et al.  The Khintchine inequalities and martingale expanding sphere of their action , 1995 .

[32]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[33]  M. Talagrand.  New concentration inequalities in product spaces , 1996 .

[34]  M. Rudelson.  Random Vectors in the Isotropic Position , 1996, math/9608208.

[35]  S. Mallat,et al.  Adaptive greedy approximations , 1997 .

[36]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[37]  M. Talagrand.  Selecting a proportion of characters , 1998 .

[38]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[39]  H. Alzer.  Inequalities for the gamma function , 1999 .

[40]  P. Massart,et al.  About the constants in Talagrand's concentration inequalities for empirical processes , 2000 .

[41]  A. Buchholz.  Operator Khintchine inequality in non-commutative probability , 2001 .

[42]  M. Ledoux.  The concentration of measure phenomenon , 2001 .

[43]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[44]  Gabriele Steidl,et al.  Fast Fourier Transforms for Nonequispaced Data: A Tutorial , 2001 .

[45]  E. Rio,et al.  Inégalités de concentration pour les processus empiriques de classes de parties , 2001 .

[46]  Michael Elad,et al.  A generalized uncertainty principle and sparse representation in pairs of bases , 2002, IEEE Trans. Inf. Theory.

[47]  P. MassartLedoux,et al.  Concentration Inequalities Using the Entropy Method , 2002 .

[48]  P. Laguna,et al.  Signal Processing , 2002, Yearbook of Medical Informatics.

[49]  E. Rio.  Une inégalité de Bennett pour les maxima de processus empiriques , 2002 .

[50]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[51]  W. Kester.  Fast Fourier Transforms , 2003 .

[52]  Thomas Strohmer,et al.  GRASSMANNIAN FRAMES WITH APPLICATIONS TO CODING AND COMMUNICATION , 2003, math/0301135.

[53]  O. Bousquet.  Concentration Inequalities for Sub-Additive Functions Using the Entropy Method , 2003 .

[54]  A. Cohen.  Numerical Analysis of Wavelet Methods , 2003 .

[55]  Rémi Gribonval,et al.  Sparse representations in unions of bases , 2003, IEEE Trans. Inf. Theory.

[56]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[57]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[58]  R. Varga.  Geršgorin And His Circles , 2004 .

[59]  Jean-Jacques Fuchs,et al.  On sparse representations in arbitrary redundant bases , 2004, IEEE Transactions on Information Theory.

[60]  R. Vershynin.  Frame expansions with erasures: an approach through the non-commutative operator theory , 2004, math/0405566.

[61]  J. Tropp.  Recovery of short, complex linear combinations via 𝓁1 minimization , 2005, IEEE Trans. Inf. Theory.

[62]  M. Talagrand.  The Generic Chaining , 2005 .

[63]  A. Buchholz.  Optimal Constants in Khintchine Type Inequalities for Fermions, Rademachers and q-Gaussian Operators , 2005 .

[64]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[65]  H. Rauhut.  Random Sampling of Sparse Trigonometric Polynomials , 2005, math/0512642.

[66]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[67]  S. Boucheron,et al.  Moment inequalities for functions of independent random variables , 2005, math/0503651.

[68]  E. Rio,et al.  Concentration around the mean for maxima of empirical processes , 2005, math/0506594.

[69]  Ali Esmaili,et al.  Probability and Random Processes , 2005, Technometrics.

[70]  M. Rudelson,et al.  Geometric approach to error-correcting codes and reconstruction of signals , 2005, math/0502299.

[71]  S. Mendelson,et al.  On singular values of matrices with independent rows , 2006 .

[72]  D. Donoho,et al.  Counting faces of randomly-projected polytopes when the projection radically lowers dimension , 2006, math/0607364.

[73]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[74]  S. Mendelson,et al.  Uniform Uncertainty Principle for Bernoulli and Subgaussian Ensembles , 2006, math/0608665.

[75]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[76]  Michael Elad,et al.  Stable recovery of sparse overcomplete representations in the presence of noise , 2006, IEEE Transactions on Information Theory.

[77]  Joel A. Tropp,et al.  Just relax: convex programming methods for identifying sparse signals in noise , 2006, IEEE Transactions on Information Theory.

[78]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[79]  Pierre Vandergheynst,et al.  On the exponential convergence of matching pursuits in quasi-incoherent dictionaries , 2006, IEEE Transactions on Information Theory.

[80]  Richard G. Baraniuk,et al.  Random Filters for Compressive Sampling and Reconstruction , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[81]  J. Wissel,et al.  On the Best Constants in the Khintchine Inequality , 2007 .

[82]  Mark Rudelson,et al.  Sampling from large matrices: An approach through geometric functional analysis , 2005, JACM.

[83]  H. Rauhut.  On the Impossibility of Uniform Sparse Reconstruction using Greedy Methods , 2007 .

[84]  R. Gribonval,et al.  Highly sparse representations from dictionaries are unique and independent of the sparseness measure , 2007 .

[85]  E. Candès,et al.  Sparsity and incoherence in compressive sampling , 2006, math/0611957.

[86]  H. Rauhut,et al.  Learning Trigonometric Polynomials from Random Samples and Exponential Inequalities for Eigenvalues of Random Matrices , 2007, math/0701781.

[87]  Ronald A. DeVore,et al.  Deterministic constructions of compressed sensing matrices , 2007, J. Complex..

[88]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[89]  Stephen J. Wright,et al.  Toeplitz-Structured Compressed Sensing Matrices , 2007, 2007 IEEE/SP 14th Workshop on Statistical Signal Processing.

[90]  Holger Rauhut,et al.  Sparsity in Time-Frequency Representations , 2007, ArXiv.

[91]  S. Mendelson,et al.  Majorizing measures and proportional subsets of bounded orthonormal systems , 2008, 0801.3556.

[92]  M. Rudelson,et al.  On sparse reconstruction from Fourier and Gaussian measurements , 2008 .

[93]  R. DeVore,et al.  Compressed sensing and best k-term approximation , 2008 .

[94]  Pierre Vandergheynst,et al.  Dictionary Preconditioning for Greedy Algorithms , 2008, IEEE Transactions on Signal Processing.

[95]  Holger Rauhut,et al.  Random Sampling of Sparse Trigonometric Polynomials, II. Orthogonal Matching Pursuit versus Basis Pursuit , 2008, Found. Comput. Math..

[96]  Yaakov Tsaig,et al.  Fast Solution of $\ell _{1}$ -Norm Minimization Problems When the Solution May Be Sparse , 2008, IEEE Transactions on Information Theory.

[97]  I. Daubechies,et al.  Accelerated Projected Gradient Method for Linear Inverse Problems with Sparsity Constraints , 2007, 0706.4297.

[98]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .

[99]  J. Romberg,et al.  Imaging via Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[100]  Ting Sun,et al.  Single-pixel imaging via compressive sampling , 2008, IEEE Signal Process. Mag..

[101]  E. Candès.  The restricted isometry property and its implications for compressed sensing , 2008 .

[102]  J. Tropp.  On the conditioning of random subdictionaries , 2008 .

[103]  Holger Rauhut,et al.  Edinburgh Research Explorer Identification of Matrices Having a Sparse Representation , 2022 .

[104]  Holger Rauhut.  Stability Results for Random Sampling of Sparse Trigonometric Polynomials , 2008, IEEE Transactions on Information Theory.

[105]  I. Daubechies,et al.  Iteratively reweighted least squares minimization for sparse recovery , 2008, 0807.0575.

[106]  Justin K. Romberg,et al.  Compressive Sensing by Random Convolution , 2009, SIAM J. Imaging Sci..

[107]  Thomas Strohmer,et al.  High-Resolution Radar via Compressed Sensing , 2008, IEEE Transactions on Signal Processing.

[108]  Holger Rauhut,et al.  Circulant and Toeplitz matrices in compressed sensing , 2009, ArXiv.

[109]  J. Azaïs,et al.  Level Sets and Extrema of Random Processes and Fields , 2009 .

[110]  S. Foucart,et al.  Sparsest solutions of underdetermined linear systems via ℓq-minimization for 0 , 2009 .

[111]  L. Nikolova,et al.  On ψ- interpolation spaces , 2009 .

[112]  Deanna Needell,et al.  Uniform Uncertainty Principle and Signal Recovery via Regularized Orthogonal Matching Pursuit , 2007, Found. Comput. Math..

[113]  Holger Rauhut,et al.  Compressive Estimation of Doubly Selective Channels in Multicarrier Systems: Leakage Effects and Sparsity-Enhancing Processing , 2009, IEEE Journal of Selected Topics in Signal Processing.

[114]  Thomas Strohmer,et al.  Compressed Remote Sensing of Sparse Objects , 2009, SIAM J. Imaging Sci..

[115]  Holger Rauhut,et al.  The Gelfand widths of ℓp-balls for 0 , 2010, ArXiv.

[116]  Justin K. Romberg,et al.  Beyond Nyquist: Efficient Sampling of Sparse Bandlimited Signals , 2009, IEEE Transactions on Information Theory.

[117]  Deanna Needell,et al.  Signal Recovery From Incomplete and Inaccurate Measurements Via Regularized Orthogonal Matching Pursuit , 2007, IEEE Journal of Selected Topics in Signal Processing.

[118]  Robert D. Nowak,et al.  Toeplitz Compressed Sensing Matrices With Applications to Sparse Channel Estimation , 2010, IEEE Transactions on Information Theory.

[119]  Rémi Gribonval,et al.  Real versus complex null space properties for sparse vector recovery , 2010 .

[120]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[121]  Jöran Bergh,et al.  Interpolation Spaces: An Introduction , 2011 .

[122]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[123]  A.,et al.  FAST FOURIER TRANSFORMS FOR NONEQUISPACED DATA * , 2022 .