A Barzilai–Borwein-Like Iterative Half Thresholding Algorithm for the $$L_{1/2}$$L1/2 Regularized Problem

In this paper, we propose a Barzilai–Borwein-like iterative half thresholding algorithm for the $$L_{1/2}$$L1/2 regularized problem. The algorithm is closely related to the iterative reweighted minimization algorithm and the iterative half thresholding algorithm. Under mild conditions, we verify that any accumulation point of the sequence of iterates generated by the algorithm is a first-order stationary point of the $$L_{1/2}$$L1/2 regularized problem. We also prove that any accumulation point is a local minimizer of the $$L_{1/2}$$L1/2 regularized problem when additional conditions are satisfied. Furthermore, we show that the worst-case iteration complexity for finding an $$\varepsilon $$ε scaled first-order stationary point is $$O(\varepsilon ^{-2})$$O(ε-2). Preliminary numerical results show that the proposed algorithm is practically effective.

[1]  Zhe Sun,et al.  A constrained optimization reformulation and a feasible descent direction method for $$L_{1/2}$$L1/2 regularization , 2014, Comput. Optim. Appl..

[2]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[3]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[4]  Zhaosong Lu,et al.  Iterative reweighted minimization methods for $$l_p$$lp regularized unconstrained nonlinear programming , 2012, Math. Program..

[5]  Wotao Yin,et al.  FIXED-POINT CONTINUATION APPLIED TO COMPRESSED SENSING: IMPLEMENTATION AND NUMERICAL EXPERIMENTS * , 2010 .

[6]  Stephen J. Wright,et al.  Sparse Reconstruction by Separable Approximation , 2008, IEEE Transactions on Signal Processing.

[7]  L. Grippo,et al.  A nonmonotone line search technique for Newton's method , 1986 .

[8]  Marcos Raydan,et al.  The Barzilai and Borwein Gradient Method for the Large Scale Unconstrained Minimization Problem , 1997, SIAM J. Optim..

[9]  Yuhong Dai A New Analysis on the Barzilai-Borwein Gradient Method , 2013 .

[10]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[11]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[12]  Zongben Xu,et al.  Regularization: Convergence of Iterative Half Thresholding Algorithm , 2014 .

[13]  William W. Hager,et al.  Gradient-Based Methods for Sparse Recovery , 2009, SIAM J. Imaging Sci..

[14]  Yue Wang,et al.  Using the multi-living agent concept to investigate complex information systems , 2008, Science in China Series F: Information Sciences.

[15]  Xiaojun Chen,et al.  Convergence of the reweighted ℓ1 minimization algorithm for ℓ2–ℓp minimization , 2013, Computational Optimization and Applications.

[16]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[17]  Xiaojun Chen,et al.  Complexity analysis of interior point algorithms for non-Lipschitz and nonconvex minimization , 2015, Math. Program..

[18]  Y. Ye,et al.  Lower Bound Theory of Nonzero Entries in Solutions of ℓ2-ℓp Minimization , 2010, SIAM J. Sci. Comput..

[19]  Zongben Xu,et al.  $L_{1/2}$ Regularization: A Thresholding Representation Theory and a Fast Solver , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[20]  M. Lai,et al.  An Unconstrained $\ell_q$ Minimization with $0q\leq1$ for Sparse Solution of Underdetermined Linear Systems , 2011 .

[21]  Yin Zhang,et al.  Fixed-Point Continuation for l1-Minimization: Methodology and Convergence , 2008, SIAM J. Optim..

[22]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[23]  M. Lai,et al.  Improved Iteratively Reweighted Least Squares for Unconstrained Smoothed 퓁q Minimization , 2013, SIAM J. Numer. Anal..