Combining line search and trust-region methods for -minimization

ABSTRACT This study presents a new trust-region algorithm to solve the -minimization problem with applications to compressed sensing (CS) and image deblurring that will be augmented with a shrinkage operation to produce a new iteration whenever an approximated solution of the trust-region subproblem lies within one and iterate is successful, simultaneously. Otherwise, a nonmonotone Armijo-type line search strategy incorporates with shrinkage technique, which includes a convex combination of the maximum function value of some preceding iterates and the current function value. Therefore, the proposed approach takes advantages of both the effective trust-region and nonmonotone Armijo-type line search with a shrinkage operation. It is believed that selecting an appropriate shrinkage parameter according to a new procedure can improve the efficiency of our algorithm. The global convergence and the R-linear convergence rate of the proposed approach are proved for which numerical results are also reported.

[1]  Jan Vybíral,et al.  Compressed Sensing and its Applications , 2015 .

[2]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[3]  Yunhai Xiao,et al.  Nonmonotone Barzilai–Borwein Gradient Algorithm for $$\ell _{1}$$ℓ1-Regularized Nonsmooth Minimization in Compressive Sensing , 2012, J. Sci. Comput..

[4]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[5]  Masoud Ahookhosh,et al.  An effective trust-region-based approach for symmetric nonlinear systems , 2013, Int. J. Comput. Math..

[6]  Masoud Ahookhosh,et al.  A nonmonotone trust-region line search method for large-scale unconstrained optimization , 2012 .

[7]  Michael Unser,et al.  A fast iterative thresholding algorithm for wavelet-regularized deconvolution , 2007, SPIE Optical Engineering + Applications.

[8]  C. G. Broyden,et al.  The convergence of an algorithm for solving sparse nonlinear systems , 1971 .

[9]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[10]  Yin Zhang,et al.  A Fast Algorithm for Sparse Reconstruction Based on Shrinkage, Subspace Optimization, and Continuation , 2010, SIAM J. Sci. Comput..

[11]  Wotao Yin,et al.  FIXED-POINT CONTINUATION APPLIED TO COMPRESSED SENSING: IMPLEMENTATION AND NUMERICAL EXPERIMENTS * , 2010 .

[12]  Michael K. Ng,et al.  A fast minimization method for blur and multiplicative noise removal , 2013, Int. J. Comput. Math..

[13]  L. Grippo,et al.  A nonmonotone line search technique for Newton's method , 1986 .

[14]  Paul Tseng,et al.  A coordinate gradient descent method for nonsmooth separable minimization , 2008, Math. Program..

[15]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[16]  M. Saunders.  A DUAL ACTIVE-SET QUADRATIC PROGRAMMING METHOD FOR FINDING SPARSE LEAST-SQUARES SOLUTIONS , 2012 .

[17]  E. Michael Gertz,et al.  A quasi-Newton trust-region method , 2004, Math. Program..

[18]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[19]  Hong Zhu,et al.  Primal and dual alternating direction algorithms for ℓ1-ℓ1-norm minimization problems in compressive sensing , 2012, Computational Optimization and Applications.

[20]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[21]  Masoud Ahookhosh,et al.  Optimal subgradient algorithms with application to large-scale linear inverse problems , 2014, 1402.7291.

[22]  Michael P. Friedlander,et al.  Probing the Pareto Frontier for Basis Pursuit Solutions , 2008, SIAM J. Sci. Comput..

[23]  Wotao Yin,et al.  TR 0707 A Fixed-Point Continuation Method for ` 1-Regularized Minimization with Applications to Compressed Sensing , 2007 .

[24]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[25]  Robert D. Nowak,et al.  An EM algorithm for wavelet-based image restoration , 2003, IEEE Trans. Image Process..

[26]  Wotao Yin,et al.  On the convergence of an active-set method for ℓ1 minimization , 2012, Optim. Methods Softw..

[27]  C. Lemaréchal,et al.  The watchdog technique for forcing convergence in algorithms for constrained optimization , 1982 .

[28]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[29]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[30]  Kim-Chuan Toh,et al.  A coordinate gradient descent method for ℓ1-regularized convex minimization , 2011, Comput. Optim. Appl..

[31]  Emmanuel J. Candès,et al.  NESTA: A Fast and Accurate First-Order Method for Sparse Recovery , 2009, SIAM J. Imaging Sci..

[32]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[33]  Bobby Schnabel,et al.  Tensor methods for large sparse systems of nonlinear equations , 1996, Math. Program..

[34]  José M. Bioucas-Dias,et al.  A New TwIST: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration , 2007, IEEE Transactions on Image Processing.

[35]  Jorge J. Moré,et al.  Benchmarking optimization software with performance profiles , 2001, Math. Program..

[36]  Stephen J. Wright,et al.  Sparse Reconstruction by Separable Approximation , 2008, IEEE Transactions on Signal Processing.

[37]  T. Blumensath,et al.  Theory and Applications , 2011 .

[38]  Antonin Chambolle,et al.  Nonlinear wavelet image processing: variational problems, compression, and noise removal through wavelet shrinkage , 1998, IEEE Trans. Image Process..