Analysis and Algorithms for Some Compressed Sensing Models Based on L1/L2 Minimization

Recently, in a series of papers [30,36,37,39], the ratio of $\ell_1$ and $\ell_2$ norms was proposed as a sparsity inducing function for noiseless compressed sensing. In this paper, we further study properties of such model in the noiseless setting, and propose an algorithm for minimizing $\ell_1$/$\ell_2$ subject to noise in the measurements. Specifically, we show that the extended objective function (the sum of the objective and the indicator function of the constraint set) of the model in [30] satisfies the Kurdyka-Lojasiewicz (KL) property with exponent 1/2; this allows us to establish linear convergence of the algorithm proposed in [37, Eq. 11] under mild assumptions. We next extend the $\ell_1$/$\ell_2$ model to handle compressed sensing problems with noise. We establish the solution existence for some of these models under the spherical section property [35,42], and extend the algorithm in [37, Eq. 11] by incorporating moving-balls-approximation techniques [4] for solving these problems. We prove the subsequential convergence of our algorithm under mild conditions, and establish global convergence of the whole sequence generated by our algorithm by imposing additional KL and differentiability assumptions on a specially constructed potential function. Finally, we perform numerical experiments on robust compressed sensing and basis pursuit denoising with residual error measured by $ \ell_2 $ norm or Lorentzian norm via solving the corresponding $\ell_1$/$\ell_2$ models by our algorithm. Our numerical simulations show that our algorithm is able to recover the original sparse vectors with reasonable accuracy.

[1]  S. Vavasis Derivation of Compressive Sensing Theorems from the Spherical Section property , 2009 .

[2]  Akiko Takeda,et al.  A refined convergence analysis of pDCAe\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hbox {pDCA}_{e}$$\end{document , 2018, Computational Optimization and Applications.

[3]  B. Mordukhovich Variational analysis and generalized differentiation , 2006 .

[4]  Yin Zhang,et al.  Theory of Compressive Sensing via ℓ1-Minimization: a Non-RIP Analysis and Extensions , 2013 .

[5]  Michael P. Friedlander,et al.  Probing the Pareto Frontier for Basis Pursuit Solutions , 2008, SIAM J. Sci. Comput..

[6]  Brian M. Sadler,et al.  Robust compressive sensing of sparse signals: a review , 2016, EURASIP J. Adv. Signal Process..

[7]  Jack Xin,et al.  Ratio and difference of $l_1$ and $l_2$ norms and sparse representation with coherent dictionaries , 2014, Commun. Inf. Syst..

[8]  Chao Wang,et al.  A Scale Invariant Approach for Sparse Signal Recovery , 2018, SIAM J. Sci. Comput..

[9]  Bastian Goldlücke,et al.  Variational Analysis , 2014, Computer Vision, A Reference Guide.

[10]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[11]  J. Borwein,et al.  Techniques of variational analysis , 2005 .

[12]  Hédy Attouch,et al.  Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Lojasiewicz Inequality , 2008, Math. Oper. Res..

[13]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[14]  F. Giannessi Variational Analysis and Generalized Differentiation , 2006 .

[15]  Hédy Attouch,et al.  On the convergence of the proximal algorithm for nonsmooth functions involving analytic features , 2008, Math. Program..

[16]  Huynh van Ngai,et al.  Error bounds for systems of lower semicontinuous functions in Asplund spaces , 2008, Math. Program..

[17]  Zheng Chen,et al.  The multiproximal linearization method for convex composite problems , 2017, Math. Program..

[18]  Bo Wen,et al.  A proximal difference-of-convex algorithm with extrapolation , 2016, Computational Optimization and Applications.

[19]  Ting Kei Pong,et al.  Deducing Kurdyka-{\L}ojasiewicz exponent via inf-projection , 2019, 1902.03635.

[20]  Adrian S. Lewis,et al.  The [barred L]ojasiewicz Inequality for Nonsmooth Subanalytic Functions with Applications to Subgradient Dynamical Systems , 2006, SIAM J. Optim..

[21]  Marc Teboulle,et al.  A Moving Balls Approximation Method for a Class of Smooth Constrained Minimization Problems , 2010, SIAM J. Optim..

[22]  Luisa F. Polanía,et al.  Compressive sensing for ECG signals in the presence of electromyographic noise , 2012, 2012 38th Annual Northeast Bioengineering Conference (NEBEC).

[23]  Rick Chartrand,et al.  Exact Reconstruction of Sparse Signals via Nonconvex Minimization , 2007, IEEE Signal Processing Letters.

[24]  H. Tuy Convex analysis and global optimization , 1998 .

[25]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[26]  F. Clarke Optimization And Nonsmooth Analysis , 1983 .

[27]  丸山 徹 Convex Analysisの二,三の進展について , 1977 .

[28]  Marc Teboulle,et al.  A dual method for minimizing a nonsmooth objective over one smooth inequality constraint , 2016, Math. Program..

[29]  Ting Kei Pong,et al.  Convergence Rate Analysis of a Sequential Convex Programming Method with Line Search for a Class of Constrained Difference-of-Convex Optimization Problems , 2021, SIAM J. Optim..

[30]  Guoyin Li,et al.  Calculus of the Exponent of Kurdyka–Łojasiewicz Inequality and Its Applications to Linear Convergence of First-Order Methods , 2016, Foundations of Computational Mathematics.

[31]  Adrian S. Lewis,et al.  Convex Analysis And Nonlinear Optimization , 2000 .

[32]  Akiko Takeda,et al.  A refined convergence analysis of pDCA$_e$ with applications to simultaneous sparse recovery and outlier detection , 2018 .

[33]  Kenneth E. Barner,et al.  Robust Sampling and Reconstruction Methods for Sparse Signals in the Presence of Impulsive Noise , 2010, IEEE Journal of Selected Topics in Signal Processing.

[34]  Marc Teboulle,et al.  Proximal alternating linearized minimization for nonconvex and nonsmooth problems , 2013, Mathematical Programming.

[35]  D. Donoho,et al.  Atomic Decomposition by Basis Pursuit , 2001 .

[36]  Werner Dinkelbach On Nonlinear Fractional Programming , 1967 .

[37]  HeQi,et al.  Computing Sparse Representation in a Highly Coherent Dictionary Based on Difference of L1 and L2 , 2015 .

[38]  Edouard Pauwels,et al.  Majorization-Minimization Procedures and Convergence of SQP Methods for Semi-Algebraic and Tame Programs , 2014, Math. Oper. Res..

[39]  Benar Fux Svaiter,et al.  Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods , 2013, Math. Program..

[40]  Chao Wang,et al.  Limited-angle CT reconstruction via the L1/L2 minimization , 2020, SIAM J. Imaging Sci..

[41]  Jack Xin,et al.  Computing Sparse Representation in a Highly Coherent Dictionary Based on Difference of $$L_1$$L1 and $$L_2$$L2 , 2015, J. Sci. Comput..

[42]  Xiaojun Chen,et al.  Penalty Methods for a Class of Non-Lipschitz Optimization Problems , 2016, SIAM J. Optim..

[43]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[44]  Jacques A. Ferland,et al.  Revisiting Dinkelbach-type algorithms for generalized fractional programs , 2008 .