Sparse Signal Processing with Frame Theory

Many emerging applications involve sparse signals, and their processing is a subject of active research. We desire a large class of sensing matrices which allow the user to discern important properties of the measured sparse signal. Of particular interest are matrices with the restricted isometry property (RIP). RIP matrices are known to enable efficient and stable reconstruction of sufficiently sparse signals, but the deterministic construction of such matrices has proven very difficult. In this thesis, we discuss this matrix design problem in the context of a growing field of study known as frame theory. In the first two chapters, we build large families of equiangular tight frames and full spark frames, and we discuss their relationship to RIP matrices as well as their utility in other aspects of sparse signal processing. In Chapter 3, we pave the road to deterministic RIP matrices, evaluating various techniques to demonstrate RIP, and making interesting connections with graph theory and number theory. We conclude in Chapter 4 with a coherence-based alternative to RIP, which provides near-optimal probabilistic guarantees for various aspects of sparse signal processing while at the same time admitting a whole host of deterministic constructions.

[1]  R. Balan,et al.  On signal reconstruction without phase , 2006 .

[2]  Jelena Kovacevic,et al.  Real, tight frames with maximal robustness to erasures , 2005, Data Compression Conference.

[3]  Ronald A. DeVore,et al.  Deterministic constructions of compressed sensing matrices , 2007, J. Complex..

[4]  Dustin G. Mixon,et al.  Constructing finite frames of a given spectrum and set of lengths , 2011, 1106.0921.

[5]  Rémi Gribonval,et al.  Sparse representations in unions of bases , 2003, IEEE Trans. Inf. Theory.

[6]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .

[7]  J WainwrightMartin.  Sharp thresholds for high-dimensional and noisy sparsity recovery using l1-constrained quadratic programming (Lasso) , 2009 .

[8]  Pascal Koiran,et al.  On the Certification of the Restricted Isometry Property , 2011, ArXiv.

[9]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[10]  V. Paulsen,et al.  Optimal frames for erasures , 2004 .

[11]  A. Robert Calderbank,et al.  Frame coherence and sparse signal processing , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[12]  N. J. A. Sloane,et al.  Packing Lines, Planes, etc.: Packings in Grassmannian Spaces , 1996, Exp. Math..

[13]  Holger Rauhut.  Stability Results for Random Sampling of Sparse Trigonometric Polynomials , 2008, IEEE Transactions on Information Theory.

[14]  Joseph M. Renes.  Equiangular Tight Frames from Paley Tournaments , 2004 .

[15]  Leslie G. Valiant,et al.  NP is as easy as detecting unique solutions , 1985, STOC '85.

[16]  Vladimir N. Temlyakov,et al.  On the size of incoherent systems , 2011, J. Approx. Theory.

[17]  Mátyás A. Sustik,et al.  On the existence of equiangular tight frames , 2007 .

[18]  C. Colbourn,et al.  Handbook of Combinatorial Designs , 2006 .

[19]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[20]  Pierre Moulin,et al.  Regular Simplex Fingerprints and Their Optimality Properties , 2005, IEEE Transactions on Information Forensics and Security.

[21]  Dustin G. Mixon,et al.  The Road to Deterministic Matrices with the Restricted Isometry Property , 2012, Journal of Fourier Analysis and Applications.

[22]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[23]  Fan Chung Graham,et al.  Quasi-random graphs , 1988, Comb..

[24]  D. M. Appleby.  Symmetric informationally complete–positive operator valued measures and the extended Clifford group , 2005 .

[25]  A. Hayashi,et al.  Reexamination of optimal quantum state estimation of pure states (5 pages) , 2004, quant-ph/0410207.

[26]  Stephen J. Dilworth,et al.  Explicit constructions of RIP matrices and related problems , 2010, ArXiv.

[27]  Gongguo Tang,et al.  Performance Analysis for Sparse Support Recovery , 2009, IEEE Transactions on Information Theory.

[28]  Mahdad Khatirinejad,et al.  On Weyl-Heisenberg orbits of equiangular lines , 2008 .

[29]  R. Calderbank,et al.  Chirp sensing codes: Deterministic compressed sensing measurements for fast recovery , 2009 .

[30]  Thomas Strohmer,et al.  A note on equiangular tight frames , 2008 .

[31]  A. Singer.  Angular Synchronization by Eigenvectors and Semidefinite Programming. , 2009, Applied and computational harmonic analysis.

[32]  O. Christensen.  An introduction to frames and Riesz bases , 2002 .

[33]  Yoshio Mimura.  A construction of spherical 2-design , 1990, Graphs Comb..

[34]  Joe Kilian,et al.  A Note on the Limits of Collusion-Resistant Watermarks , 1999, EUROCRYPT.

[35]  Dustin G. Mixon,et al.  Fingerprinting With Equiangular Tight Frames , 2011, IEEE Transactions on Information Theory.

[36]  John E. Hopcroft,et al.  Complexity of Computer Computations , 1974, IFIP Congress.

[37]  S. Delvaux,et al.  Rank-deficient submatrices of Fourier matrices , 2008 .

[38]  Holger Rauhut,et al.  Edinburgh Research Explorer Identification of Matrices Having a Sparse Representation , 2022 .

[39]  S. Kay.  Fundamentals of statistical signal processing: estimation theory , 1993 .

[40]  David Duncan,et al.  Equiangular tight frames and fourth root seidel matrices , 2010 .

[41]  Bernhard G. Bodmann,et al.  Fast algorithms for signal reconstruction without phase , 2007, SPIE Optical Engineering + Applications.

[42]  S. Godsill,et al.  Multi-Gabor dictionaries for audio time-frequency analysis , 2001, Proceedings of the 2001 IEEE Workshop on the Applications of Signal Processing to Audio and Acoustics (Cat. No.01TH8575).

[43]  Jean-Jacques Fuchs,et al.  Sparsity and uniqueness for some specific under-determined linear systems , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[44]  Peter G. Casazza,et al.  Constructing tight fusion frames , 2011 .

[45]  Bernhard G. Bodmann,et al.  The road to equal-norm Parseval frames , 2010 .

[46]  P. Casazza,et al.  The Kadison–Singer Problem in mathematics and engineering , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[47]  Dustin G. Mixon,et al.  Equiangular tight frame fingerprinting codes , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[48]  J. Tropp.  Norms of Random Submatrices and Sparse Approximation , 2008 .

[49]  Waheed Uz,et al.  NEW INFORMATION PROCESSING THEORY AND METHODS FOR EXPLOITING SPARSITY IN WIRELESS SYSTEMS , 2009 .

[50]  Elwyn R. Berlekamp.  The Weight Enumerators for Certain Subcodes of the Second Order Binary Reed-Muller Codes , 1970, Inf. Control..

[51]  William O. Alltop,et al.  Complex sequences with low periodic correlations (Corresp.) , 1980, IEEE Trans. Inf. Theory.

[52]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[53]  Dustin G. Mixon,et al.  Two are better than one: Fundamental parameters of frame coherence , 2011, 1103.0435.

[54]  Stephen D. Cohen.  CLIQUE NUMBERS OF PALEY GRAPHS , 1988 .

[55]  Peter G. Casazza,et al.  Fourier Transforms of Finite Chirps , 2006, EURASIP J. Adv. Signal Process..

[56]  J. J. Seidel,et al.  Equilateral point sets in elliptic geometry , 1966 .

[57]  Joel Friedman,et al.  A proof of Alon's second eigenvalue conjecture and related problems , 2004, ArXiv.

[58]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[59]  Noga Alon,et al.  Approximating the cut-norm via Grothendieck's inequality , 2004, STOC '04.

[60]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[61]  Dustin G. Mixon,et al.  Full Spark Frames , 2011, 1110.3548.

[62]  Joseph M. Renes,et al.  Symmetric informationally complete quantum measurements , 2003, quant-ph/0310075.

[63]  M. Rudelson,et al.  On sparse reconstruction from Fourier and Gaussian measurements , 2008 .

[64]  John Langford,et al.  Multi-Label Prediction via Compressed Sensing , 2009, NIPS.

[65]  J. Tropp.  On the conditioning of random subdictionaries , 2008 .

[66]  S. T. McCormick,et al.  A combinatorial approach to some sparse matrix problems , 1983 .

[67]  Dominic Welsh,et al.  On the Vector Representation of Matroids , 1970 .

[68]  J. Seidel.  Strongly regular graphs with (-1, 1, 0) adjacency matrix having eigenvalue 3 , 1968 .

[69]  M. Murty.  Ramanujan Graphs , 1965 .

[70]  V. V. Yurinskii.  Exponential inequalities for sums of random vectors , 1976 .

[71]  R. J. R. Abel,et al.  BIBDs with Small Block Size , 2006 .

[72]  E. Chong,et al.  Robust measurement design for detecting sparse signals: Equiangular uniform tight frames and grassmannian packings , 2010, Proceedings of the 2010 American Control Conference.

[73]  Alan M. Frieze,et al.  Random graphs , 2006, SODA '06.

[74]  S. Graham,et al.  Lower Bounds for Least Quadratic Non-Residues , 1990 .

[75]  Peter G. Casazza,et al.  Equal-Norm Tight Frames with Erasures , 2003, Adv. Comput. Math..

[76]  Martin J. Wainwright,et al.  Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _{1}$ -Constrained Quadratic Programming (Lasso) , 2009, IEEE Transactions on Information Theory.

[77]  Dustin G. Mixon,et al.  Steiner equiangular tight frames , 2010, 1009.5730.

[78]  David L. Donoho,et al.  Observed universality of phase transitions in high-dimensional geometry, with implications for modern data analysis and signal processing , 2009, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[79]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[80]  L. M. M.-T..  Theory of Probability , 1929, Nature.

[81]  Peter G. Casazza,et al.  Real equiangular frames , 2008, 2008 42nd Annual Conference on Information Sciences and Systems.

[82]  S. Chowla,et al.  On Difference Sets. , 1949, Proceedings of the National Academy of Sciences of the United States of America.

[83]  Yonina C. Eldar,et al.  Coherence-Based Performance Guarantees for Estimating a Sparse Vector Under Random Noise , 2009, IEEE Transactions on Signal Processing.

[84]  Robert W. Heath,et al.  Designing structured tight frames via an alternating projection method , 2005, IEEE Transactions on Information Theory.

[85]  Lloyd R. Welch,et al.  Lower bounds on the maximum cross correlation of signals (Corresp.) , 1974, IEEE Trans. Inf. Theory.

[86]  Thomas Strohmer,et al.  High-Resolution Radar via Compressed Sensing , 2008, IEEE Transactions on Signal Processing.

[87]  Matthew Fickus.  Maximally Equiangular Frames and Gauss Sums , 2009 .

[88]  CONSTRUCTING A LARGE FAMILY OF EQUIANGULAR TIGHT FRAMES , 2010 .

[89]  Emmanuel J. Candès,et al.  PhaseLift: Exact and Stable Signal Recovery from Magnitude Measurements via Convex Programming , 2011, ArXiv.

[90]  N. Linial,et al.  Expander Graphs and their Applications , 2006 .

[91]  Min Wu,et al.  Anti-collusion forensics of multimedia fingerprinting using orthogonal modulation , 2005, IEEE Transactions on Image Processing.

[92]  Bhaskar D. Rao,et al.  Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm , 1997, IEEE Trans. Signal Process..

[93]  John J. Benedetto,et al.  Finite Normalized Tight Frames , 2003, Adv. Comput. Math..

[94]  M. Rudelson,et al.  Non-asymptotic theory of random matrices: extreme singular values , 2010, 1003.2990.

[95]  Thomas Strohmer,et al.  GRASSMANNIAN FRAMES WITH APPLICATIONS TO CODING AND COMMUNICATION , 2003, math/0301135.

[96]  T. Tao.  An uncertainty principle for cyclic groups of prime order , 2003, math/0308286.

[97]  William O. Alltop,et al.  Complex sequences with low periodic correlations , 1980 .

[98]  A. J. Scott,et al.  Symmetric informationally complete positive-operator-valued measures: A new computer study , 2010 .

[99]  Preeti Singh.  Equiangular tight frames and signature sets in groups , 2010 .

[100]  S. Szarek,et al.  Chapter 8 - Local Operator Theory, Random Matrices and Banach Spaces , 2001 .

[101]  H. Carfantan,et al.  A Sparsity-Based Method for the Estimation of Spectral Lines From Irregularly Sampled Data , 2007, IEEE Journal of Selected Topics in Signal Processing.

[102]  R. Balan,et al.  Painless Reconstruction from Magnitudes of Frame Coefficients , 2009 .

[103]  W. Hoeffding.  Probability Inequalities for sums of Bounded Random Variables , 1963 .

[104]  J. Lawrence,et al.  Linear Independence of Gabor Systems in Finite Dimensional Vector Spaces , 2005 .

[105]  H. Hanani,et al.  On steiner systems , 1964 .

[106]  Noga Alon,et al.  Eigenvalues and expanders , 1986, Comb..

[107]  John J. Benedetto,et al.  Geometric Properties of Grassmannian Frames for and , 2006, EURASIP J. Adv. Signal Process..

[108]  Robert E. Tarjan,et al.  Resistance of digital watermarks to collusive attacks , 1998, Proceedings. 1998 IEEE International Symposium on Information Theory (Cat. No.98CH36252).

[109]  Deepti Kalra,et al.  Complex equiangular cyclic frames and erasures , 2006 .

[110]  Bhaskar D. Rao,et al.  Sparse Bayesian learning for basis selection , 2004, IEEE Transactions on Signal Processing.

[111]  Michael Elad,et al.  Stable recovery of sparse overcomplete representations in the presence of noise , 2006, IEEE Transactions on Information Theory.

[112]  Gerald M. Masson,et al.  Lower Bounds on Crosspoints in Concentrators , 1982, IEEE Transactions on Computers.

[113]  R. Peralta.  On the distribution of quadratic residues and nonresidues modulo a prime number , 1992 .

[114]  John J. Benedetto,et al.  Geometric Properties of Grassmannian Frames for R 2 and R 3 , 2004 .

[115]  JuttenChristian,et al.  A fast approach for overcomplete sparse decomposition based on smoothed l0 norm , 2009 .

[116]  Peter J. Cameron,et al.  Strongly regular graphs , 2003 .

[117]  Robert D. Nowak,et al.  Toeplitz Compressed Sensing Matrices With Applications to Sparse Channel Estimation , 2010, IEEE Transactions on Information Theory.

[118]  Ingemar J. Cox,et al.  Secure spread spectrum watermarking for multimedia , 1997, IEEE Trans. Image Process..

[119]  P. Massart,et al.  Adaptive estimation of a quadratic functional by model selection , 2000 .

[120]  V. Paulsen,et al.  Frames, graphs and erasures , 2004, math/0406134.

[121]  E. Candès.  The restricted isometry property and its implications for compressed sensing , 2008 .

[122]  R. Duffin,et al.  A class of nonharmonic Fourier series , 1952 .

[123]  Peter G. Casazza,et al.  Auto-tuning unit norm frames , 2010, 1009.5562.

[124]  Guang Gong,et al.  A new binary sequence family with low correlation and large size , 2006, IEEE Transactions on Information Theory.

[125]  Bernhard G. Bodmann,et al.  Equiangular tight frames from complex Seidel matrices containing cube roots of unity , 2008, 0805.2014.

[126]  A. Robert Calderbank,et al.  Why Gabor frames? Two fundamental measures of coherence and their role in model selection , 2010, Journal of Communications and Networks.

[127]  Richard M. Karp,et al.  Reducibility Among Combinatorial Problems , 1972, 50 Years of Integer Programming.

[128]  Dilip V. Sarwate.  Meeting the Welch Bound with Equality , 1998, SETA.

[129]  Nate Strawn,et al.  Finite Frame Varieties: Nonsingular Points, Tangent Spaces, and Explicit Local Parameterizations , 2011 .

[130]  G. Bennett.  Probability Inequalities for the Sum of Independent Random Variables , 1962 .

[131]  I. Daubechies,et al.  PAINLESS NONORTHOGONAL EXPANSIONS , 1986 .

[132]  Robert D. Nowak,et al.  Compressive Sampling for Signal Detection , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[133]  E. Candès,et al.  Near-ideal model selection by ℓ1 minimization , 2008, 0801.0345.

[134]  Noga Alon,et al.  The Probabilistic Method , 2015, Fundamentals of Ramsey Theory.

[135]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[136]  Dániel Marx,et al.  A parameterized view on matroid optimization problems , 2009, Theor. Comput. Sci..

[137]  Richard G. Baraniuk,et al.  Signal Processing With Compressive Measurements , 2010, IEEE Journal of Selected Topics in Signal Processing.

[138]  Helen J. Elwood,et al.  Complex equiangular Parseval frames and Seidel matrices containing $p$th roots of unity , 2010 .

[139]  J. J. Seidel,et al.  A SURVEY OF TWO-GRAPHS , 1976 .

[140]  Peter G. Casazza,et al.  Optimally Sparse Frames , 2010, IEEE Transactions on Information Theory.

[141]  Georgios B. Giannakis,et al.  Achieving the Welch bound with difference sets , 2005, IEEE Transactions on Information Theory.

[142]  Shayne Waldron,et al.  On the construction of equiangular frames from graphs , 2009 .

[143]  Noga Alon,et al.  Problems and results in extremal combinatorics--I , 2003, Discret. Math..

[144]  A. A. Makhnev,et al.  Strongly regular graphs with λ=1 , 1988 .

[145]  P. Stevenhagen,et al.  Chebotarëv and his density theorem , 1996 .