A Unified Study on $L_1$ over $L_2$ Minimization

In this paper, we carry out a unified study for L1 over L2 sparsity promoting models, which are widely used in the regime of coherent dictionaries for recovering sparse nonnegative/arbitrary signal. First, we provide the exact recovery condition on both the constrained and the unconstrained models for a broad set of signals. Next, we prove the solution existence of these L1/L2 models under the assumption that the null space of the measurement matrix satisfies the s-spherical section property. Then by deriving an analytical solution for the proximal operator of the L1/L2 with nonnegative constraint, we develop a new alternating direction method of multipliers based method (ADMM p ) to solve the unconstrained model. We establish its global convergence to a d-stationary solution (sharpest stationary) and its local linear convergence under certain conditions. Numerical simulations on two specific applications confirm the superior of ADMM p over the state-of-the-art methods in sparse recovery. ADMM p reduces computational time by about 95% ∼ 99% while achieving a much higher accuracy compared to commonly used scaled gradient projection method for wavelength misalignment problem. Index Terms Sparse recovery, the ratio of L1 over L2, alternating direction method of multipliers, coherent dictionary, dstationarity

[1]  Hédy Attouch,et al.  On the convergence of the proximal algorithm for nonsmooth functions involving analytic features , 2008, Math. Program..

[2]  Guoyin Li,et al.  Calculus of the Exponent of Kurdyka–Łojasiewicz Inequality and Its Applications to Linear Convergence of First-Order Methods , 2016, Foundations of Computational Mathematics.

[3]  Xiangfeng Wang,et al.  Asynchronous Distributed ADMM for Large-Scale Optimization—Part I: Algorithm and Convergence Analysis , 2015, IEEE Transactions on Signal Processing.

[4]  Warren Hare,et al.  Computing proximal points of nonconvex functions , 2008, Math. Program..

[5]  Jack Xin,et al.  Computational Aspects of Constrained L 1-L 2 Minimization for Compressive Sensing , 2015, MCO.

[6]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[7]  Rob Fergus,et al.  Blind deconvolution using a normalized sparsity measure , 2011, CVPR 2011.

[8]  Byonghyo Shim,et al.  Optimal Restricted Isometry Condition of Normalized Sampling Matrices for Exact Sparse Recovery With Orthogonal Least Squares , 2021, IEEE Transactions on Signal Processing.

[9]  Guoyin Li,et al.  Global Convergence of Splitting Methods for Nonconvex Composite Optimization , 2014, SIAM J. Optim..

[10]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[11]  Patrik O. Hoyer,et al.  Non-negative sparse coding , 2002, Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing.

[12]  Jack Xin,et al.  Ratio and difference of $l_1$ and $l_2$ norms and sparse representation with coherent dictionaries , 2014, Commun. Inf. Syst..

[13]  Jack Xin,et al.  A Method for Finding Structured Sparse Solutions to Nonnegative Least Squares Problems with Applications , 2013, SIAM J. Imaging Sci..

[14]  Hui Ji,et al.  Image deconvolution using a characterization of sharp images in wavelet domain , 2012 .

[15]  Jong-Shi Pang,et al.  Computing B-Stationary Points of Nonsmooth DC Programs , 2015, Math. Oper. Res..

[16]  Chao Wang,et al.  Limited-angle CT reconstruction via the L1/L2 minimization , 2020, SIAM J. Imaging Sci..

[17]  Lei Wu,et al.  A Barzilai–Borwein-Like Iterative Half Thresholding Algorithm for the $$L_{1/2}$$L1/2 Regularized Problem , 2016, J. Sci. Comput..

[18]  F. Clarke Optimization And Nonsmooth Analysis , 1983 .

[19]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[20]  Min Tao,et al.  On the Linear Convergence to Weak/Standard d-Stationary Points of DCA-Based Algorithms for Structured Nonsmooth DC Programming , 2021, Journal of Optimization Theory and Applications.

[21]  J. Pesquet,et al.  SPOQ $\ell _p$-Over-$\ell _q$ Regularization for Sparse Signal Recovery Applied to Mass Spectrometry , 2020, IEEE Transactions on Signal Processing.

[22]  Zhi-Quan Luo,et al.  Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems , 2014, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[23]  Jong-Shi Pang,et al.  Decomposition Methods for Computing Directional Stationary Solutions of a Class of Nonsmooth Nonconvex Optimization Problems , 2018, SIAM J. Optim..

[24]  Liaoyuan Zeng,et al.  Analysis and Algorithms for Some Compressed Sensing Models Based on L1/L2 Minimization , 2020, SIAM J. Optim..

[25]  A. Hoffman On approximate solutions of systems of linear inequalities , 1952 .

[26]  Anthony Man-Cho So,et al.  Understanding Notions of Stationarity in Nonsmooth Optimization: A Guided Tour of Various Constructions of Subdifferential for Nonsmooth Functions , 2020, IEEE Signal Processing Magazine.

[27]  Yin Zhang,et al.  Theory of Compressive Sensing via ℓ1-Minimization: a Non-RIP Analysis and Extensions , 2013 .

[28]  Adrian S. Lewis,et al.  The [barred L]ojasiewicz Inequality for Nonsmooth Subanalytic Functions with Applications to Subgradient Dynamical Systems , 2006, SIAM J. Optim..

[29]  Hing Cheung So,et al.  Learning Proximal Operator Methods for Nonconvex Sparse Recovery with Theoretical Guarantee , 2020, IEEE Transactions on Signal Processing.

[30]  Wotao Yin,et al.  Improved Iteratively Reweighted Least Squares for Unconstrained Smoothed 퓁q Minimization , 2013, SIAM J. Numer. Anal..

[31]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[32]  S. Vavasis Derivation of Compressive Sensing Theorems from the Spherical Section property , 2009 .

[33]  MINIMIZATION OF L1 OVER L2 FOR SPARSE SIGNAL , 2020 .

[34]  Chao Wang,et al.  A Scale Invariant Approach for Sparse Signal Recovery , 2018, SIAM J. Sci. Comput..