Reweighted 1-Minimization for Sparse Solutions to Underdetermined Linear Systems

Numerical experiments have indicated that the reweighted $\ell_1$-minimization performs exceptionally well in locating sparse solutions of underdetermined linear systems of equations. We show that reweighted $\ell_1$-methods are intrinsically associated with the minimization of the so-called merit functions for sparsity, which are essentially concave approximations to the cardinality function. Based on this observation, we further show that a family of reweighted $\ell_1$-algorithms can be systematically derived from the perspective of concave optimization through the linearization technique. In order to conduct a unified convergence analysis for this family of algorithms, we introduce the concept of the range space property (RSP) of a matrix and prove that if its adjoint has this property, the reweighted $\ell_1$-algorithm can find a sparse solution to the underdetermined linear system provided that the merit function for sparsity is properly chosen. In particular, some convergence conditions for the Can...

[1]  Xiaojun Chen,et al.  Convergence of Reweighted ' 1 Minimization Algorithms and Unique Solution of Truncated ' p Minimization , 2010 .

[2]  Stephen P. Boyd,et al.  Portfolio optimization with linear and fixed transaction costs , 2007, Ann. Oper. Res..

[3]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[4]  E.J. Candes.  Compressive Sampling , 2022 .

[5]  Stephen P. Boyd,et al.  Log-det heuristic for matrix rank minimization with applications to Hankel and Euclidean distance matrices , 2003, Proceedings of the 2003 American Control Conference, 2003..

[6]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[7]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[8]  Tony F. Chan,et al.  Color TV: total variation methods for restoration of vector-valued images , 1998, IEEE Trans. Image Process..

[9]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[10]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.

[11]  I. Daubechies,et al.  Iteratively reweighted least squares minimization for sparse recovery , 2008, 0807.0575.

[12]  R. DeVore,et al.  Compressed sensing and best k-term approximation , 2008 .

[13]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[14]  Yoram Bresler,et al.  A new algorithm for computing sparse solutions to linear inverse problems , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[15]  Rick Chartrand,et al.  Exact Reconstruction of Sparse Signals via Nonconvex Minimization , 2007, IEEE Signal Processing Letters.

[16]  S. Foucart,et al.  Sparsest solutions of underdetermined linear systems via ℓq-minimization for 0 , 2009 .

[17]  Rémi Gribonval,et al.  Restricted Isometry Constants Where $\ell ^{p}$ Sparse Recovery Can Fail for $0≪ p \leq 1$ , 2009, IEEE Transactions on Information Theory.

[18]  Weiyu Xu,et al.  Null space conditions and thresholds for rank minimization , 2011, Math. Program..

[19]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[20]  J. Tropp,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, Commun. ACM.

[21]  Deanna Needell,et al.  Noisy signal recovery via iterative reweighted L1-minimization , 2009, 2009 Conference Record of the Forty-Third Asilomar Conference on Signals, Systems and Computers.

[22]  P. Holland,et al.  Robust regression using iteratively reweighted least-squares , 1977 .

[23]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[24]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[25]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[26]  Panos M. Pardalos,et al.  Introduction to Global Optimization , 2000, Introduction to Global Optimization.

[27]  D. Donoho,et al.  Uncertainty principles and signal recovery , 1989 .

[28]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[29]  Ming-Jun Lai,et al.  An Unconstrained ℓq Minimization with 0 , 2011, SIAM J. Optim..

[30]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[31]  H. L. Taylor,et al.  Deconvolution with the l 1 norm , 1979 .

[32]  Shu-Cherng Fang,et al.  Constructing Generalized Mean Functions Using Convex Functions with Regularity Conditions , 2006, SIAM J. Optim..

[33]  Bhaskar D. Rao,et al.  Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm , 1997, IEEE Trans. Signal Process..

[34]  Davies Rémi Gribonval.  Restricted Isometry Constants Where Lp Sparse Recovery Can Fail for 0 , 2008 .

[35]  David P. Wipf,et al.  Iterative Reweighted 1 and 2 Methods for Finding Sparse Solutions , 2010, IEEE J. Sel. Top. Signal Process..

[36]  Michael Elad,et al.  From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images , 2009, SIAM Rev..

[37]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[38]  Aswin C. Sankaranarayanan,et al.  Compressive Sensing , 2008, Computer Vision, A Reference Guide.

[39]  Stephen P. Boyd,et al.  Rank minimization and applications in system theory , 2004, Proceedings of the 2004 American Control Conference.

[40]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[41]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[42]  Yin Zhang,et al.  Theory of Compressive Sensing via ℓ1-Minimization: a Non-RIP Analysis and Extensions , 2013 .

[43]  Yun-Bin Zhao,et al.  Approximation Theory of Matrix Rank Minimization and Its Application to Quadratic Equations , 2010, 1010.0851.