Iterative log thresholding

Sparse reconstruction approaches using the re-weighted ℓ1-penalty have been shown, both empirically and theoretically, to provide a significant improvement in recovering sparse signals in comparison to the ℓ1-relaxation. However, numerical optimization of such penalties involves solving problems with ℓ1-norms in the objective many times. Using the direct link of reweighted ℓ1-penalties to the concave log-regularizer for sparsity, we derive a simple proximal-like algorithm for the log-regularized formulation. The proximal splitting step of the algorithm has a closed form solution, and we call the algorithm log-thresholding in analogy to soft thresholding for the ℓ1-penalty. We establish convergence results, and demonstrate that log-thresholding provides more accurate sparse reconstructions compared to both soft and hard thresholding. Furthermore, the approach can be directly extended to optimization over matrices with penalty for rank (i.e. the nuclear norm penalty and its re-weighted version), where we suggest a singular-value log-thresholding approach.

[1]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[2]  T. Hastie,et al.  SparseNet: Coordinate Descent With Nonconvex Penalties , 2011, Journal of the American Statistical Association.

[3]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[4]  Stephen P. Boyd,et al.  A rank minimization heuristic with application to minimum order system approximation , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[5]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[6]  Deanna Needell,et al.  Noisy signal recovery via iterative reweighted L1-minimization , 2009, 2009 Conference Record of the Forty-Third Asilomar Conference on Signals, Systems and Computers.

[7]  Qiuying Lin,et al.  Sparsity and nonconvex nonsmooth optimization , 2009 .

[8]  Arian Maleki,et al.  Coherence analysis of iterative thresholding algorithms , 2009, 2009 47th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[9]  D. Hunter,et al.  Optimization Transfer Using Surrogate Objective Functions , 2000 .

[10]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[11]  Weiyu Xu,et al.  Weighted ℓ1 minimization for sparse recovery with prior information , 2009, 2009 IEEE International Symposium on Information Theory.

[12]  Stéphane Canu,et al.  Recovering Sparse Signals With a Certain Family of Nonconvex Penalties and DC Programming , 2009, IEEE Transactions on Signal Processing.

[13]  T. Blumensath,et al.  Iterative Thresholding for Sparse Approximations , 2008 .

[14]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[15]  Dmitry M. Malioutov,et al.  Homotopy continuation for sparse signal representation , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[16]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[17]  Rick Chartrand,et al.  A new generalized thresholding algorithm for inverse problems with sparsity constraints , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[18]  Bin Dong,et al.  Fast Linearized Bregman Iteration for Compressive Sensing and Sparse Denoising , 2011, ArXiv.

[19]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[20]  Michael P. Friedlander,et al.  Probing the Pareto Frontier for Basis Pursuit Solutions , 2008, SIAM J. Sci. Comput..