The Barzilai and Borwein Gradient Method with Nonmonotone Line Search for Nonsmooth Convex Optimization Problems

The Barzilai and Borwein gradient algorithm has received a great deal of attention in recent decades since it is simple and effective for smooth optimization problems. Whether can it be extended to solve nonsmooth problems? In this paper, we answer this question positively. The Barzilai and Borwein gradient algorithm combined with a nonmonotone line search technique is proposed for nonsmooth convex minimization. The global convergence of the given algorithm is established under suitable conditions. Numerical results show that this method is efficient.

[1]  Liping Zhang,et al.  A new trust region algorithm for nonsmooth convex minimization , 2007, Appl. Math. Comput..

[2]  William W. Hager,et al.  A Nonmonotone Line Search Technique and Its Application to Unconstrained Optimization , 2004, SIAM J. Optim..

[3]  Changyu Wang,et al.  Convergence properties of nonmonotone spectral projected gradient methods , 2005 .

[4]  J. M. Martínez,et al.  Spectral Gradient Methods for Linearly Constrained Optimization , 2005 .

[5]  Masao Fukushima,et al.  trust region method for nonsmooth convex optimization , 2005 .

[6]  Luigi Grippo,et al.  Nonmonotone Globalization Techniques for the Barzilai-Borwein Gradient Method , 2002, Comput. Optim. Appl..

[7]  L. Liao,et al.  R-linear convergence of the Barzilai and Borwein gradient method , 2002 .

[8]  W. Hager,et al.  The cyclic Barzilai-–Borwein method for unconstrained optimization , 2006 .

[9]  相良 信子.  A Trust Region Method for Nonsmooth Convex Optimization , 2004 .

[10]  Roger Fletcher,et al.  Projected Barzilai-Borwein methods for large-scale box-constrained quadratic programming , 2005, Numerische Mathematik.

[11]  Jochem Zowe,et al.  A Version of the Bundle Idea for Minimizing a Nonsmooth Function: Conceptual Idea, Convergence Analysis, Numerical Results , 1992, SIAM J. Optim..

[12]  Ya-Xiang Yuan,et al.  Analysis of monotone gradient methods , 2005 .

[13]  L. Qi.  Regular Pseudo-Smooth NCP and BVIP Functions and Globally and Quadratically Convergent Generalized Newton Methods for Complementarity and Variational Inequality Problems , 1999 .

[14]  L. Liao,et al.  An Analysis of the Barzilai and Borwein Gradient Method for Unsymmetric Linear Equations , 2005 .

[15]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[16]  K. Kiwiel.  Methods of Descent for Nondifferentiable Optimization , 1985 .

[17]  Liqun Qi,et al.  A new method for nonsmooth convex optimization , 1998 .

[18]  Defeng Sun,et al.  Solving Karush--Kuhn--Tucker Systems via the Trust Region and the Conjugate Gradient Methods , 2003, SIAM J. Optim..

[19]  Liqun Qi,et al.  A nonsmooth version of Newton's method , 1993, Math. Program..

[20]  Z. Wei,et al.  Convergence analysis of a proximal newton method 1 , 1996 .

[21]  José Mario Martínez,et al.  Nonmonotone Spectral Projected Gradient Methods on Convex Sets , 1999, SIAM J. Optim..

[22]  Krzysztof C. Kiwiel,et al.  Proximal level bundle methods for convex nondifferentiable optimization, saddle-point problems and variational inequalities , 1995, Math. Program..

[23]  Zhensheng Yu.  Solving bound constrained optimization via a new nonmonotone spectral projected gradient method , 2008 .

[24]  Yi Zhou,et al.  A descent spectral conjugate gradient method for impulse noise removal , 2010, Appl. Math. Lett..

[25]  J. Frédéric Bonnans,et al.  A family of variable metric proximal methods , 1995, Math. Program..

[26]  Yi Qin,et al.  A multivariate spectral projected gradient method for bound constrained optimization , 2011, J. Comput. Appl. Math..

[27]  Jan Vlcek,et al.  A bundle-Newton method for nonsmooth unconstrained minimization , 1998, Math. Program..

[28]  J. M. Martínez,et al.  A Spectral Conjugate Gradient Method for Unconstrained Optimization , 2001 .

[29]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[30]  L. Qi,et al.  A General Approach to Convergence Properties of Some Methods for Nonsmooth Convex Optimization , 1998 .

[31]  P. Wolfe.  Note on a method of conjugate subgradients for minimizing nondifferentiable functions , 1974 .

[32]  Yuhong Dai.  Alternate step gradient method , 2003 .

[33]  Liqun Qi,et al.  Convergence Analysis of Some Methods for Minimizing a Nonsmooth Convex Function , 1998 .

[34]  J. Pang,et al.  A Trust Region Method for Constrained Nonsmooth Equations , 1994 .

[35]  Claude Lemaréchal,et al.  Convergence of some algorithms for convex minimization , 1993, Math. Program..

[36]  Yunhai Xiao,et al.  Spectral gradient projection method for monotone nonlinear equations with convex constraints , 2009 .

[37]  M. Raydan.  On the Barzilai and Borwein choice of steplength for the gradient method , 1993 .

[38]  Masao Fukushima,et al.  A Globally and Superlinearly Convergent Algorithm for Nonsmooth Convex Minimization , 1996, SIAM J. Optim..

[39]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[40]  Le Han,et al.  Multivariate spectral gradient method for unconstrained optimization , 2008, Appl. Math. Comput..

[41]  Guanghui Liu,et al.  Global Convergence Analysis of a New Nonmonotone BFGS Algorithm on Convex Objective Functions , 1997, Comput. Optim. Appl..

[42]  A. Tits,et al.  Nonmonotone line search for minimax problems , 1993 .

[43]  Hongchao Zhang,et al.  Adaptive Two-Point Stepsize Gradient Algorithm , 2001, Numerical Algorithms.

[44]  Dong Wang,et al.  Notes on the Dai-Yuan-Yuan modified spectral gradient method , 2010, J. Comput. Appl. Math..

[45]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[46]  Jin Yun Yuan,et al.  Modified Two-Point Stepsize Gradient Methods for Unconstrained Optimization , 2002, Comput. Optim. Appl..

[47]  Marcos Raydan,et al.  The Barzilai and Borwein Gradient Method for the Large Scale Unconstrained Minimization Problem , 1997, SIAM J. Optim..

[48]  Krzysztof C. Kiwiel,et al.  Proximity control in bundle methods for convex nondifferentiable minimization , 1990, Math. Program..

[49]  C. Lemaréchal.  Nondifferentiable optimization , 1989 .

[50]  Wenyu Sun,et al.  Trust region algorithm for nonsmooth optimization , 1997 .

[51]  Richard A. Tapia,et al.  A unified approach to global convergence of trust region methods for nonsmooth optimization , 1995, Math. Program..

[52]  Weijun Zhou,et al.  Spectral gradient projection method for solving nonlinear monotone equations , 2006 .

[53]  J. M. Martínez,et al.  Gradient Method with Retards and Generalizations , 1998 .

[54]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[55]  L. Grippo,et al.  A nonmonotone line search technique for Newton's method , 1986 .