An efficient conjugate gradient method with strong convergence properties for non-smooth optimization

In this paper, we introduce an efficient conjugate gradient method for solving nonsmooth optimization problems by using the Moreau-Yosida regularization approach. The search directions generated by our proposed procedure satisfy the sufficient descent property, and more importantly, belong to a suitable trust region.  Our proposed method is globally convergent under mild assumptions. Our numerical comparative results on a collection of test problems show the efficiency and superiority of our proposed method. We have also examined the ability and the effectiveness of our approach for solving some real-world engineering problems from image processing field. The results confirm better performance of our method.

[1]  Xin Zhang,et al.  A Modified Nonlinear Conjugate Gradient Algorithm for Large-Scale Nonsmooth Convex Optimization , 2020, J. Optim. Theory Appl..

[2]  Masao Fukushima,et al.  A Globally and Superlinearly Convergent Algorithm for Nonsmooth Convex Minimization , 1996, SIAM J. Optim..

[3]  Adil M. Bagirov,et al.  Comparing different nonsmooth minimization methods and software , 2012, Optim. Methods Softw..

[4]  D. Gleich.  TRUST REGION METHODS , 2017 .

[5]  Claude Lemaréchal,et al.  Practical Aspects of the Moreau-Yosida Regularization: Theoretical Preliminaries , 1997, SIAM J. Optim..

[6]  Gonglin Yuan,et al.  A conjugate gradient algorithm and its application in large-scale optimization problems and image restoration , 2019, Journal of Inequalities and Applications.

[7]  Neculai Andrei.  A Numerical Study on Efficiency and Robustness of Some Conjugate Gradient Algorithms for Large-scale Unconstrained Optimization , 2013 .

[8]  W. Hager,et al.  A SURVEY OF NONLINEAR CONJUGATE GRADIENT METHODS , 2005 .

[9]  Liping Zhang,et al.  A new trust region algorithm for nonsmooth convex minimization , 2007, Appl. Math. Comput..

[10]  Yuzhen Lu,et al.  Out-of-focus Blur: Image De-blurring , 2017, ArXiv.

[11]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[12]  Dianne P. O'Leary,et al.  Deblurring Images: Matrices, Spectra and Filtering , 2006, J. Electronic Imaging.

[13]  Gonglin Yuan,et al.  The Barzilai and Borwein Gradient Method with Nonmonotone Line Search for Nonsmooth Convex Optimization Problems , 2012 .

[14]  F. Meng,et al.  On Second-Order Properties of the Moreau–Yosida Regularization for Constrained Nonsmooth Convex Programs , 2004 .

[15]  Claude Lemaréchal,et al.  Convergence of some algorithms for convex minimization , 1993, Math. Program..

[16]  Lue Li,et al.  A trust region algorithm with adaptive cubic regularization methods for nonsmooth convex minimization , 2012, Comput. Optim. Appl..

[17]  Qiong Li.  Conjugate gradient type methods for the nondifferentiable convex minimization , 2013, Optim. Lett..

[18]  Kaisa Miettinen,et al.  New limited memory bundle method for large-scale nonsmooth optimization , 2004, Optim. Methods Softw..

[19]  Masoud Fatemi.  A new efficient conjugate gradient method for unconstrained optimization , 2016, J. Comput. Appl. Math..

[20]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[21]  Yaping Hu.  Multivariate Spectral Gradient Algorithm for Nonsmooth ConvexOptimization Problems , 2015 .

[22]  Yong Li,et al.  A Modified Hestenes and Stiefel Conjugate Gradient Algorithm for Large-Scale Nonsmooth Minimizations and Nonlinear Equations , 2015, Journal of Optimization Theory and Applications.

[23]  Liqun Qi,et al.  Convergence Analysis of Some Algorithms for Solving Nonsmooth Equations , 1993, Math. Oper. Res..

[24]  Gonglin Yuan,et al.  Gradient trust region algorithm with limited memory BFGS update for nonsmooth convex minimization , 2012, Computational Optimization and Applications.

[25]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[26]  M. Fukushima,et al.  Globally Convergent BFGS Method for Nonsmooth Convex Optimization1 , 2000 .

[27]  C. Storey,et al.  Efficient generalized conjugate gradient algorithms, part 1: Theory , 1991 .

[28]  Boris Polyak.  The conjugate gradient method in extremal problems , 1969 .

[29]  Masao Fukushima,et al.  trust region method for nonsmooth convex optimization , 2005 .

[30]  Guoyin Li,et al.  A modified Polak-Ribière-Polyak conjugate gradient algorithm for nonsmooth convex programs , 2014, J. Comput. Appl. Math..

[31]  L. Liao,et al.  New Conjugacy Conditions and Related Nonlinear Conjugate Gradient Methods , 2001 .

[32]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[33]  Jin-Bao Jian,et al.  Improved Fletcher-Reeves and Dai-Yuan conjugate gradient methods with the strong Wolfe line search , 2019, J. Comput. Appl. Math..

[34]  Ya-Xiang Yuan,et al.  A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property , 1999, SIAM J. Optim..

[35]  Masao Fukushima,et al.  A descent algorithm for nonsmooth convex optimization , 1984, Math. Program..

[36]  Xiaojun Chen,et al.  Proximal quasi-Newton methods for nondifferentiable convex optimization , 1999, Math. Program..