Weak Stability of ࡁ1-minimization Methods in Sparse Data Reconstruction

As one of the most plausible convex optimization methods for sparse data reconstruction, $\ell_1$-minimization plays a fundamental role in the development of sparse optimization theory. The stability of this method has been addressed in the literature under various assumptions such as restricted isometry property (RIP), null space property (NSP), and mutual coherence. In this paper, we propose a unified means to develop the so-called weak stability theory for $\ell_1$-minimization methods under the condition called weak range space property of a transposed design matrix, which turns out to be a necessary and sufficient condition for the standard $\ell_1$-minimization method to be weakly stable in sparse data reconstruction. The reconstruction error bounds established in this paper are measured by the so-called Robinson's constant. We also provide a unified weak stability result for standard $\ell_1$-minimization under several existing compressed-sensing matrix properties. In particular, the weak stability of $\ell_1$-minimization under the constant-free range space property of order $k$ of the transposed design matrix is established for the first time in this paper. Different from the existing analysis, we utilize the classic Hoffman's Lemma concerning the error bound of linear systems as well as the Dudley's theorem concerning the polytope approximation of the unit $\ell_2$-ball to show that $\ell_1$-minimization is robustly and weakly stable in recovering sparse data from inaccurate measurements.

[1]  Anru Zhang,et al.  Sparse Representation of a Polytope and Recovery of Sparse Signals and Low-Rank Matrices , 2013, IEEE Transactions on Information Theory.

[2]  Yun-Bin Zhao,et al.  A New Computational Method for the Sparsest Solutions to Systems of Linear Equations , 2015, SIAM J. Optim..

[3]  H. L. Taylor,et al.  Deconvolution with the l 1 norm , 1979 .

[4]  Michael Elad,et al.  Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing , 2010 .

[5]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[6]  Ming Yan,et al.  One condition for solution uniqueness and robustness of both l1-synthesis and l1-analysis minimizations , 2013, Adv. Comput. Math..

[7]  Lie Wang,et al.  New Bounds for Restricted Isometry Constants , 2009, IEEE Transactions on Information Theory.

[8]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[9]  S. M. Robinson Bounds for error in the solution set of a perturbed linear program , 1973 .

[10]  Holger Rauhut,et al.  A Mathematical Introduction to Compressive Sensing , 2013, Applied and Numerical Harmonic Analysis.

[11]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[12]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[13]  Yin Zhang,et al.  Theory of Compressive Sensing via ℓ1-Minimization: a Non-RIP Analysis and Extensions , 2013 .

[14]  E. Candès The restricted isometry property and its implications for compressed sensing , 2008 .

[15]  Jan-Olov Strömberg,et al.  On the Theorem of Uniform Recovery of Structured Random Matrices , 2012, ArXiv.

[16]  Song Li,et al.  New bounds on the restricted isometry constant δ2k , 2011 .

[17]  Andrew R. Barron,et al.  A Better Approximation for Balls , 2000 .

[18]  Jianping Hu,et al.  Rewighted L1-minimization for sparse solutions to underdetermined linear systems , 2013, 2013 6th International Congress on Image and Signal Processing (CISP).

[19]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[20]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[21]  Hui Zhang,et al.  Necessary and Sufficient Conditions of Solution Uniqueness in 1-Norm Minimization , 2012, Journal of Optimization Theory and Applications.

[22]  Rongrong Wang,et al.  The gap between the null space property and the restricted isomety property , 2015, ArXiv.

[23]  Olvi L. Mangasarian,et al.  Machine Learning via Polyhedral Concave Minimization , 1996 .

[24]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[25]  Chunlei Xu,et al.  1-Bit compressive sensing: Reformulation and RRSP-based sign recovery theory , 2014, Science China Mathematics.

[26]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[27]  Mark D. Plumbley On Polar Polytopes and the Recovery of Sparse Representations , 2007, IEEE Trans. Inf. Theory.

[28]  Anru Zhang,et al.  Sharp RIP bound for sparse signal and low-rank matrix recovery , 2013 .

[29]  Jean-Jacques Fuchs,et al.  On sparse representations in arbitrary redundant bases , 2004, IEEE Transactions on Information Theory.

[30]  B. Logan,et al.  Signal recovery and the large sieve , 1992 .

[31]  Qiyu Sun,et al.  Recovery of sparsest signals via ℓq-minimization , 2010, ArXiv.

[32]  Michael Elad,et al.  From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images , 2009, SIAM Rev..

[33]  Duan Li,et al.  Reweighted 1-Minimization for Sparse Solutions to Underdetermined Linear Systems , 2012, SIAM J. Optim..

[34]  J. CandesE.,et al.  Near-Optimal Signal Recovery From Random Projections , 2006 .

[35]  Michael Elad,et al.  Stable recovery of sparse overcomplete representations in the presence of noise , 2006, IEEE Transactions on Information Theory.

[36]  Yun-Bin Zhao,et al.  RSP-Based Analysis for Sparsest and Least $\ell_1$-Norm Solutions to Underdetermined Linear Systems , 2013, IEEE Transactions on Signal Processing.

[37]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[38]  Yun-Bin Zhao,et al.  Equivalence and Strong Equivalence Between the Sparsest and Least $$\ell _1$$ℓ1-Norm Nonnegative Solutions of Linear Systems and Their Applications , 2013, 1312.4163.

[39]  R. DeVore,et al.  Compressed sensing and best k-term approximation , 2008 .

[40]  T. Blumensath,et al.  Theory and Applications , 2011 .

[41]  O. Mangasarian Minimum-support solutions of polyhedral concave programs * , 1999 .

[42]  Trevor Hastie,et al.  Statistical Learning with Sparsity: The Lasso and Generalizations , 2015 .

[43]  S. Levy,et al.  Reconstruction of a sparse spike train from a portion of its spectrum and application to high-resolution deconvolution , 1981 .