A compact formulation for the l21 mixed-norm minimization problem

We present an equivalent, compact reformulation of the ℓ2,1 mixed-norm minimization problem for joint sparse signal reconstruction from multiple measurement vectors (MMVs). The reformulation builds upon a compact parameterization, which models the row-norms of the sparse signal representation as parameters of interest, resulting in a significant reduction of the MMV problem size. Given the sparse vector of row-norms, the joint sparse signal can be computed from the MMVs in closed form. For the special case of uniform linear sampling, we present an extension of the compact formulation for gridless parameter estimation by means of semidefinite programming. Furthermore, we derive in this case from our compact problem formulation the exact equivalence between the ℓ2,1 mixed-norm minimization and the atomic-norm minimization.

[1]  Emmanuel J. Candès,et al.  Quantitative Robust Uncertainty Principles and Optimally Sparse Decompositions , 2004, Found. Comput. Math..

[2]  Lihua Xie,et al.  On Gridless Sparse Methods for Line Spectral Estimation From Complete and Incomplete Data , 2014, IEEE Transactions on Signal Processing.

[3]  D. Donoho Superresolution via sparsity constraints , 1992 .

[4]  J. Tropp Algorithms for simultaneous sparse approximation. Part II: Convex relaxation , 2006, Signal Process..

[5]  Dmitry M. Malioutov,et al.  A sparse signal reconstruction perspective for source localization with sensor arrays , 2005, IEEE Transactions on Signal Processing.

[6]  Thomas Strohmer,et al.  General Deviants: An Analysis of Perturbations in Compressed Sensing , 2009, IEEE Journal of Selected Topics in Signal Processing.

[7]  Yonina C. Eldar,et al.  Rank Awareness in Joint Sparse Recovery , 2010, IEEE Transactions on Information Theory.

[8]  Petre Stoica,et al.  MUSIC, maximum likelihood and Cramer-Rao bound , 1988, ICASSP-88., International Conference on Acoustics, Speech, and Signal Processing.

[9]  Petre Stoica,et al.  SPICE and LIKES: Two hyperparameter-free methods for sparse-parameter estimation , 2012, Signal Process..

[10]  Stephen J. Wright Coordinate descent algorithms , 2015, Mathematical Programming.

[11]  Jie Chen,et al.  Theoretical Results on Sparse Representations of Multiple-Measurement Vectors , 2006, IEEE Transactions on Signal Processing.

[12]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[13]  Gongguo Tang,et al.  Near minimax line spectral estimation , 2013, 2013 47th Annual Conference on Information Sciences and Systems (CISS).

[14]  M. Viberg,et al.  Two decades of array signal processing research: the parametric approach , 1996, IEEE Signal Process. Mag..

[15]  Joel A. Tropp,et al.  Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit , 2006, Signal Process..

[16]  R. Kumaresan,et al.  Estimation of frequencies of multiple sinusoids: Making linear prediction perform like maximum likelihood , 1982, Proceedings of the IEEE.

[17]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[18]  Emmanuel J. Candès,et al.  Super-Resolution from Noisy Data , 2012, Journal of Fourier Analysis and Applications.

[19]  Bhaskar D. Rao,et al.  Support Recovery of Sparse Signals in the Presence of Multiple Measurement Vectors , 2011, IEEE Transactions on Information Theory.

[20]  Jian Li,et al.  Weighted SPICE: A unifying approach for hyperparameter-free sparse estimation , 2014, Digit. Signal Process..

[21]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[22]  Lihua Xie,et al.  Exact Joint Sparse Frequency Recovery via Optimization Methods , 2014, 1405.6585.

[23]  Montse Nájar,et al.  Sparse covariance fitting for direction of arrival estimation , 2012, EURASIP J. Adv. Signal Process..

[24]  Jian Li,et al.  SPICE: A Sparse Covariance-Based Estimation Method for Array Processing , 2011, IEEE Transactions on Signal Processing.

[25]  C. Carathéodory Über den variabilitätsbereich der fourier’schen konstanten von positiven harmonischen funktionen , 1911 .

[26]  Mashud Hyder,et al.  Direction-of-Arrival Estimation Using a Mixed � , 2010 .

[27]  Otto Toeplitz,et al.  Zur Theorie der quadratischen und bilinearen Formen von unendlichvielen Veränderlichen , 1911 .

[28]  Yuejie Chi,et al.  Off-the-Grid Line Spectrum Denoising and Estimation With Multiple Measurement Vectors , 2014, IEEE Transactions on Signal Processing.

[29]  Stephen P. Boyd,et al.  Graph Implementations for Nonsmooth Convex Programs , 2008, Recent Advances in Learning and Control.

[30]  Katya Scheinberg,et al.  Noname manuscript No. (will be inserted by the editor) Efficient Block-coordinate Descent Algorithms for the Group Lasso , 2022 .

[31]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[32]  R. O. Schmidt,et al.  Multiple emitter location and signal Parameter estimation , 1986 .

[33]  Jian Li,et al.  New Method of Sparse Parameter Estimation in Separable Models and Its Use for Spectral Analysis of Irregularly Sampled Data , 2011, IEEE Transactions on Signal Processing.

[34]  Håkan Hjalmarsson,et al.  A Note on the SPICE Method , 2012, IEEE Transactions on Signal Processing.

[35]  Emmanuel J. Cand Towards a Mathematical Theory of Super-Resolution , 2012 .

[36]  Stephen J. Wright,et al.  Simultaneous Variable Selection , 2005, Technometrics.

[37]  P. Stoica,et al.  The stochastic CRB for array processing: a textbook derivation , 2001, IEEE Signal Processing Letters.

[38]  Gongguo Tang,et al.  Sparse recovery over continuous dictionaries-just discretize , 2013, 2013 Asilomar Conference on Signals, Systems and Computers.

[39]  A. Robert Calderbank,et al.  Sensitivity to Basis Mismatch in Compressed Sensing , 2010, IEEE Transactions on Signal Processing.

[40]  M. Lai,et al.  The null space property for sparse recovery from multiple measurement vectors , 2011 .

[41]  Pablo A. Parrilo,et al.  The Convex Geometry of Linear Inverse Problems , 2010, Foundations of Computational Mathematics.

[42]  Benjamin Recht,et al.  Atomic norm denoising with applications to line spectral estimation , 2011, Allerton.

[43]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[44]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[45]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[46]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[47]  J. Högbom,et al.  APERTURE SYNTHESIS WITH A NON-REGULAR DISTRIBUTION OF INTERFEROMETER BASELINES. Commentary , 1974 .

[48]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[49]  Pál Turán,et al.  Über den Zusammenhang der Extremen von Harmonischen Funktionen mit Ihren Koeffizienten und Über den Picard—Landauschen Satz , 1970 .

[50]  Jong Chul Ye,et al.  Compressive MUSIC: A Missing Link Between Compressive Sensing and Array Signal Processing , 2010, ArXiv.

[51]  Justin K. Romberg,et al.  Beyond Nyquist: Efficient Sampling of Sparse Bandlimited Signals , 2009, IEEE Transactions on Information Theory.

[52]  Bhaskar D. Rao,et al.  Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm , 1997, IEEE Trans. Signal Process..

[53]  Tapan K. Sarkar,et al.  Matrix pencil method for estimating parameters of exponentially damped/undamped sinusoids in noise , 1990, IEEE Trans. Acoust. Speech Signal Process..

[54]  Mostafa Kaveh,et al.  Sparse Spatial Spectral Estimation: A Covariance Fitting Algorithm, Performance and Regularization , 2013, IEEE Transactions on Signal Processing.

[55]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[56]  Marc E. Pfetsch,et al.  A Compact Formulation for the $\ell _{2,1}$ Mixed-Norm Minimization Problem , 2016, IEEE Transactions on Signal Processing.

[57]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[58]  M. Kowalski Sparse regression using mixed norms , 2009 .

[59]  Bhaskar D. Rao,et al.  Sparse solutions to linear inverse problems with multiple measurement vectors , 2005, IEEE Transactions on Signal Processing.

[60]  Parikshit Shah,et al.  Compressed Sensing Off the Grid , 2012, IEEE Transactions on Information Theory.

[61]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[62]  Petre Stoica,et al.  Connection between SPICE and Square-Root LASSO for sparse parameter estimation , 2014, Signal Process..

[63]  Igal Bilik,et al.  Spatial Compressive Sensing for Direction-of-Arrival Estimation With Bias Mitigation Via Expected Likelihood , 2013, IEEE Transactions on Signal Processing.