An Alternative Lagrange-Dual Based Algorithm for Sparse Signal Reconstruction

In this correspondence, we propose a new Lagrange-dual reformulation associated with an l1 -norm minimization problem for sparse signal reconstruction. There are two main advantages of our proposed approach. First, the number of the variables in the reformulated optimization problem is much smaller than that in the original problem when the dimension of measurement vector is much less than the size of the original signals; Second, the new problem is smooth and convex, and hence it can be solved by many state of the art gradient-type algorithms efficiently. The efficiency and performance of the proposed algorithm are validated via theoretical analysis as well as some illustrative numerical examples.

[1]  A. Atkinson.  Subset Selection in Regression , 1992 .

[2]  Meihua Li,et al.  Improved iterative algorithm for sparse object reconstruction and its performance evaluation with micro-CT data , 2004, IEEE Transactions on Nuclear Science.

[3]  Michael Elad,et al.  A generalized uncertainty principle and sparse representation in pairs of bases , 2002, IEEE Trans. Inf. Theory.

[4]  K. Kreutz-Delgado,et al.  Efficient backward elimination algorithm for sparse signal representation using overcomplete dictionaries , 2002, IEEE Signal Processing Letters.

[5]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[6]  Jean-Jacques Fuchs,et al.  Recovery of exact sparse representations in the presence of noise , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[7]  Katta G. Murty,et al.  Nonlinear Programming Theory and Algorithms , 2007, Technometrics.

[8]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[9]  Mark A. Iwen,et al.  A deterministic sub-linear time sparse fourier algorithm via non-adaptive compressed sensing methods , 2007, SODA '08.

[10]  Bhaskar D. Rao,et al.  Backward sequential elimination for sparse vector subset selection , 2001, Signal Process..

[11]  D. Donoho.  For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .

[12]  J. Fuchs.  More on sparse representations in arbitrary bases , 2003 .

[13]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[14]  Masashi Sugiyama,et al.  Dual-Augmented Lagrangian Method for Efficient Sparse Reconstruction , 2009, IEEE Signal Processing Letters.

[15]  Shuicheng Yan,et al.  Semi-supervised Learning by Sparse Representation , 2009, SDM.

[16]  Joel A. Tropp,et al.  Just relax: convex programming methods for identifying sparse signals in noise , 2006, IEEE Transactions on Information Theory.

[17]  O. Mangasarian,et al.  NONLINEAR PERTURBATION OF LINEAR PROGRAMS , 1979 .

[18]  E. Candès.  The restricted isometry property and its implications for compressed sensing , 2008 .

[19]  Guillermo Sapiro,et al.  Simultaneous structure and texture image inpainting , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[20]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[21]  Yaakov Tsaig,et al.  Breakdown of equivalence between the minimal l1-norm solution and the sparsest solution , 2006, Signal Process..

[22]  S. Mallat,et al.  Adaptive greedy approximations , 1997 .

[23]  Dan Roth,et al.  Learning a Sparse Representation for Object Detection , 2002, ECCV.

[24]  E.J. Candes.  Compressive Sampling , 2022 .

[25]  Emmanuel J. Cand.  The Restricted Isometry Property and Its Implications for Compressed Sensing , 2008 .

[26]  Yoram Bresler,et al.  On the Optimality of the Backward Greedy Algorithm for the Subset Selection Problem , 2000, SIAM J. Matrix Anal. Appl..

[27]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[28]  Keigo Hirakawa,et al.  Effective separation of sparse and non-sparse image features for denoising , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[29]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[30]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[31]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Kim-Chuan Toh,et al.  A coordinate gradient descent method for ℓ1-regularized convex minimization , 2011, Comput. Optim. Appl..

[33]  Yaakov Tsaig,et al.  Extensions of compressed sensing , 2006, Signal Process..

[34]  Juan Carlos Niebles,et al.  Unsupervised Learning of Human Action Categories Using Spatial-Temporal Words , 2006, BMVC.

[35]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[36]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[37]  Sudipto Guha,et al.  Near-optimal sparse fourier representations via sampling , 2002, STOC '02.

[38]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[39]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[40]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[41]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[42]  Jean-Jacques Fuchs,et al.  On sparse representations in arbitrary redundant bases , 2004, IEEE Transactions on Information Theory.

[43]  Barak A. Pearlmutter,et al.  Blind Source Separation by Sparse Decomposition in a Signal Dictionary , 2001, Neural Computation.

[44]  Robert D. Nowak,et al.  A bound optimization approach to wavelet-based image deconvolution , 2005, IEEE International Conference on Image Processing 2005.

[45]  Jorge Nocedal,et al.  A Limited Memory Algorithm for Bound Constrained Optimization , 1995, SIAM J. Sci. Comput..

[46]  Stephen J. Wright,et al.  Computational Methods for Sparse Solution of Linear Inverse Problems , 2010, Proceedings of the IEEE.

[47]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[48]  Stephen P. Boyd,et al.  An Interior-Point Method for Large-Scale $\ell_1$-Regularized Least Squares , 2007, IEEE Journal of Selected Topics in Signal Processing.