Convergence of descent method without line search

Line search method is sometimes a useful and efficient technique for solving unconstrained optimization problems, especially for solving small and middle scale problems. However, A line search procedure is necessary at each iteration, leading to a significant amount of computation. In order to reduce the evaluations of objective functions and gradients, the line search procedure should be avoided in algorithm design. In this paper we propose a new descent method without line search for unconstrained optimization problems. The algorithm is required to estimate some parameters at each iteration. We analyze theoretically the global convergence of the new algorithm under mild conditions. These theoretical conclusions can help us in designing new efficient methods for optimization problems.

[1]  L. Grippo,et al.  A truncated Newton method with nonmonotone line search for unconstrained optimization , 1989 .

[2]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[3]  Jie Sun,et al.  Global convergence of a two-parameter family of conjugate gradient methods without line search , 2002 .

[4]  Kok Lay Teo,et al.  Optimization and control with applications , 2005 .

[5]  Luigi Grippo,et al.  A globally convergent version of the Polak-Ribière conjugate gradient method , 1997, Math. Program..

[6]  L. Grippo,et al.  A class of nonmonotone stabilization methods in unconstrained optimization , 1991 .

[7]  Philippe L. Toint,et al.  An Assessment of Nonmonotone Linesearch Techniques for Unconstrained Optimization , 1996, SIAM J. Sci. Comput..

[8]  M. Todd.  On Convergence Properties of Algorithms for Unconstrained Minimization , 1989 .

[9]  Jorge Nocedal,et al.  Global Convergence Properties of Conjugate Gradient Methods for Optimization , 1992, SIAM J. Optim..

[10]  H. Y. Huang,et al.  Quadratically convergent algorithms and one-dimensional search schemes , 1973 .

[11]  Jorge J. Moré,et al.  Testing Unconstrained Optimization Software , 1981, TOMS.

[12]  M. N. Vrahatisa,et al.  A class of gradient unconstrained minimization algorithms with adaptive stepsize , 1999 .

[13]  L. Armijo.  Minimization of functions having Lipschitz continuous first partial derivatives. , 1966 .

[14]  Jiapu Zhang,et al.  Global Convergence of Conjugate Gradient Methods without Line Search , 2001, Ann. Oper. Res..

[15]  Marcos Raydan,et al.  The Barzilai and Borwein Gradient Method for the Large Scale Unconstrained Minimization Problem , 1997, SIAM J. Optim..

[16]  Z. J. Shi,et al.  New Inexact Line Search Method for Unconstrained Optimization , 2005 .

[17]  A. Goldstein.  On Steepest Descent , 1965 .

[18]  L. C. W. Dixon.  Conjugate Directions without Linear Searches , 1973 .

[19]  L. Liao,et al.  R-linear convergence of the Barzilai and Borwein gradient method , 2002 .

[20]  M. Raydan.  On the Barzilai and Borwein choice of steplength for the gradient method , 1993 .

[21]  L. Grippo,et al.  A nonmonotone line search technique for Newton's method , 1986 .

[22]  P. Wolfe.  Convergence Conditions for Ascent Methods. II: Some Corrections , 1971 .

[23]  M. N. Vrahatis,et al.  A New Unconstrained Optimization Method for Imprecise Function and Gradient Values , 1996 .

[24]  P. Wolfe.  Convergence Conditions for Ascent Methods. II , 1969 .

[25]  Wenyu Sun,et al.  Global convergence of nonmonotone descent methods for unconstrained optimization problems , 2002 .

[26]  Zhen-Jun Shi,et al.  Convergence of line search methods for unconstrained optimization , 2004, Appl. Math. Comput..

[27]  Wenyu Sun,et al.  An adaptive approach of conic trust-region method for unconstrained optimization problems , 2005 .

[28]  Ya-Xiang Yuan,et al.  Convergence properties of Beale-Powell restart algorithm , 1998 .

[29]  Philippe L. Toint,et al.  Non-monotone trust-region algorithms for nonlinear optimization subject to convex constraints , 1997, Math. Program..

[30]  A. I. Cohen.  Stepsize analysis for descent methods , 1973, CDC 1973.

[31]  Ioannis K. Argyros,et al.  Weak sufficient convergence conditions and applications for newton methods , 2004 .