Exact Joint Sparse Frequency Recovery via Optimization Methods

Frequency recovery/estimation from discrete samples of superimposed sinusoidal signals is a classic yet important problem in statistical signal processing. Its research has recently been advanced by atomic norm techniques that exploit signal sparsity, work directly on continuous frequencies, and completely resolve the grid mismatch problem of previous compressed sensing methods. In this paper, we investigate the frequency recovery problem in the presence of multiple measurement vectors (MMVs) which share the same frequency components, termed as joint sparse frequency recovery and arising naturally from array processing applications. To study the advantage of MMVs, we first propose an ℓ2,0 norm like approach by exploiting joint sparsity and show that the number of recoverable frequencies can be increased except in a trivial case. While the resulting optimization problem is shown to be rank minimization that cannot be practically solved, we then propose an MMV atomic norm approach that is a convex relaxation and can be viewed as a continuous counterpart of the ℓ2,1 norm method. We show that this MMV atomic norm approach can be solved by semidefinite programming. We also provide theoretical results showing that the frequencies can be exactly recovered under appropriate conditions. The above results either extend the MMV compressed sensing results from the discrete to the continuous setting or extend the recent super-resolution and continuous compressed sensing framework from the single to the multiple measurement vectors case. Extensive simulation results are provided to validate our theoretical findings and they also imply that the proposed MMV atomic norm approach can improve the performance in terms of reduced number of required measurements and/or relaxed frequency separation condition.

[1]  Gongguo Tang,et al.  Near minimax line spectral estimation , 2013, 2013 47th Annual Conference on Information Sciences and Systems (CISS).

[2]  Gongguo Tang,et al.  Atomic Norm Denoising With Applications to Line Spectral Estimation , 2012, IEEE Transactions on Signal Processing.

[3]  M. Viberg,et al.  Two decades of array signal processing research: the parametric approach , 1996, IEEE Signal Process. Mag..

[4]  Cishen Zhang,et al.  A Discretization-Free Sparse and Parametric Approach for Linear Array Signal Processing , 2013, IEEE Transactions on Signal Processing.

[5]  Randolph L. Moses,et al.  Dynamic Dictionary Algorithms for Model Order and Parameter Estimation , 2013, IEEE Transactions on Signal Processing.

[6]  Yuxin Chen,et al.  Robust Spectral Compressed Sensing via Structured Matrix Completion , 2013, IEEE Transactions on Information Theory.

[7]  Qiang Fu,et al.  Compressed Sensing of Complex Sinusoids: An Approach Based on Dictionary Refinement , 2012, IEEE Transactions on Signal Processing.

[8]  Yonina C. Eldar,et al.  Average Case Analysis of Multichannel Sparse Recovery Using Convex Relaxation , 2009, IEEE Transactions on Information Theory.

[9]  A. Schaeffer Inequalities of A. Markoff and S. Bernstein for polynomials and related functions , 1941 .

[10]  Dmitry M. Malioutov,et al.  A sparse signal reconstruction perspective for source localization with sensor arrays , 2005, IEEE Transactions on Signal Processing.

[11]  Massimo Fornasier,et al.  Recovery Algorithms for Vector-Valued Data with Joint Sparsity Constraints , 2008, SIAM J. Numer. Anal..

[12]  Thomas Kailath,et al.  ESPRIT-estimation of signal parameters via rotational invariance techniques , 1989, IEEE Trans. Acoust. Speech Signal Process..

[13]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[14]  Dustin G. Mixon,et al.  Full Spark Frames , 2011, 1110.3548.

[15]  Joel A. Tropp,et al.  User-Friendly Tail Bounds for Sums of Random Matrices , 2010, Found. Comput. Math..

[16]  Yoram Bresler,et al.  Subspace Methods for Joint Sparse Recovery , 2010, IEEE Transactions on Information Theory.

[17]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[18]  R. Cooke Real and Complex Analysis , 2011 .

[19]  Jong Chul Ye,et al.  Compressive MUSIC: Revisiting the Link Between Compressive Sensing and Array Signal Processing , 2012, IEEE Trans. Inf. Theory.

[20]  Yonina C. Eldar,et al.  Robust Recovery of Signals From a Structured Union of Subspaces , 2008, IEEE Transactions on Information Theory.

[21]  Michael P. Friedlander,et al.  Theoretical and Empirical Results for Recovery From Multiple Measurements , 2009, IEEE Transactions on Information Theory.

[22]  Emmanuel J. Candès,et al.  Towards a Mathematical Theory of Super‐resolution , 2012, ArXiv.

[23]  Lihua Xie,et al.  On Gridless Sparse Methods for Line Spectral Estimation From Complete and Incomplete Data , 2014, IEEE Transactions on Signal Processing.

[24]  Yonina C. Eldar,et al.  Direction of Arrival Estimation Using Co-Prime Arrays: A Super Resolution Viewpoint , 2013, IEEE Transactions on Signal Processing.

[25]  Emmanuel J. Candès,et al.  Super-Resolution from Noisy Data , 2012, Journal of Fourier Analysis and Applications.

[26]  M. Kowalski Sparse regression using mixed norms , 2009 .

[27]  Bhaskar D. Rao,et al.  Sparse solutions to linear inverse problems with multiple measurement vectors , 2005, IEEE Transactions on Signal Processing.

[28]  H. Rauhut,et al.  Atoms of All Channels, Unite! Average Case Analysis of Multi-Channel Sparse Recovery Using Greedy Algorithms , 2008 .

[29]  Yonina C. Eldar,et al.  Rank Awareness in Joint Sparse Recovery , 2010, IEEE Transactions on Information Theory.

[30]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[31]  Benjamin Recht,et al.  A Simpler Approach to Matrix Completion , 2009, J. Mach. Learn. Res..

[32]  Yuejie Chi,et al.  Off-the-Grid Line Spectrum Denoising and Estimation With Multiple Measurement Vectors , 2014, IEEE Transactions on Signal Processing.

[33]  V. Pisarenko The Retrieval of Harmonics from a Covariance Function , 1973 .

[34]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[35]  Christian Jutten,et al.  Recovery of Low-Rank Matrices Under Affine Constraints via a Smoothed Rank Function , 2013, IEEE Transactions on Signal Processing.

[36]  V. Paulsen Completely Bounded Maps and Operator Algebras: Contents , 2003 .

[37]  David B. Dunson,et al.  Multitask Compressive Sensing , 2009, IEEE Transactions on Signal Processing.

[38]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[39]  Kim-Chuan Toh,et al.  SDPT3 -- A Matlab Software Package for Semidefinite Programming , 1996 .

[40]  Jie Chen,et al.  Theoretical Results on Sparse Representations of Multiple-Measurement Vectors , 2006, IEEE Transactions on Signal Processing.

[41]  Roman Vershynin,et al.  Introduction to the non-asymptotic analysis of random matrices , 2010, Compressed Sensing.

[42]  Cishen Zhang,et al.  Off-Grid Direction of Arrival Estimation Using Sparse Bayesian Inference , 2011, IEEE Transactions on Signal Processing.

[43]  Cishen Zhang,et al.  Robustly Stable Signal Recovery in Compressed Sensing With Structured Matrix Perturbation , 2011, IEEE Transactions on Signal Processing.

[44]  M. Vetterli,et al.  Sparse Sampling of Signal Innovations , 2008, IEEE Signal Processing Magazine.

[45]  Ralph Otto Schmidt,et al.  A signal subspace approach to multiple emitter location and spectral estimation , 1981 .

[46]  Yuejie Chi,et al.  Joint sparsity recovery for spectral compressed sensing , 2013, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[47]  Parikshit Shah,et al.  Compressed Sensing Off the Grid , 2012, IEEE Transactions on Information Theory.

[48]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[49]  Lihua Xie,et al.  Continuous compressed sensing with a single or multiple measurement vectors , 2014, 2014 IEEE Workshop on Statistical Signal Processing (SSP).

[50]  Jun Fang,et al.  Super-Resolution Compressed Sensing: An Iterative Reweighted Algorithm for Joint Parameter Learning and Sparse Signal Recovery , 2014, IEEE Signal Processing Letters.

[51]  U. Grenander,et al.  Toeplitz Forms And Their Applications , 1958 .

[52]  J. Kruskal Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics , 1977 .

[53]  Yonina C. Eldar,et al.  Reduce and Boost: Recovering Arbitrary Sets of Jointly Sparse Vectors , 2008, IEEE Transactions on Signal Processing.

[54]  Maryam Fazel,et al.  Iterative reweighted algorithms for matrix rank minimization , 2012, J. Mach. Learn. Res..

[55]  C. Carathéodory,et al.  Über den zusammenhang der extremen von harmonischen funktionen mit ihren koeffizienten und über den picard-landau’schen satz , 1911 .

[56]  Kaushik Mahata,et al.  Direction-of-Arrival Estimation Using a Mixed $\ell _{2,0}$ Norm Approximation , 2010, IEEE Transactions on Signal Processing.

[57]  Petre Stoica,et al.  Spectral Analysis of Signals , 2009 .

[58]  F. Gamboa,et al.  Spike detection from inaccurate samplings , 2013, 1301.5873.

[59]  Pablo A. Parrilo,et al.  The Convex Geometry of Linear Inverse Problems , 2010, Foundations of Computational Mathematics.

[60]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[61]  Jian Li,et al.  SPICE: A Sparse Covariance-Based Estimation Method for Array Processing , 2011, IEEE Transactions on Signal Processing.