Efficient line search algorithm for unconstrained optimization

A new line search algorithm for smooth unconstrained optimization is presented that requires only one gradient evaluation with an inaccurate line search and at most two gradient evaluations with an accurate line search. It terminates in finitely many operations and shares the same theoretical properties as the standard line search rules like the Armijo-Goldstein-Wolfe-Powell rules. This algorithm is especially appropriate for the situation when gradient evaluations are very expensive relative to function evaluations.

[1]  A. Goldstein.  On Steepest Descent , 1965 .

[2]  Michael A. Saunders,et al.  A note on a sufficient-decrease criterion for a non-derivative step-length procedure , 1982, Math. Program..

[3]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[4]  Allen A. Goldstein,et al.  Constructive Real Analysis , 1967 .

[5]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[6]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[7]  J. Nocedal,et al.  Global Convergence of a Class of Quasi-newton Methods on Convex Problems, Siam Some Global Convergence Properties of a Variable Metric Algorithm for Minimization without Exact Line Searches, Nonlinear Programming, Edited , 1996 .

[8]  L. Armijo.  Minimization of functions having Lipschitz continuous first partial derivatives. , 1966 .

[9]  David G. Luenberger,et al.  Linear and nonlinear programming , 1984 .

[10]  R. Brent.  Table errata: Algorithms for minimization without derivatives (Prentice-Hall, Englewood Cliffs, N. J., 1973) , 1975 .

[11]  G. R. Walsh,et al.  Methods Of Optimization , 1976 .

[12]  Jorge J. Moré,et al.  Testing Unconstrained Optimization Software , 1981, TOMS.

[13]  R. D. Murphy,et al.  Iterative solution of nonlinear equations , 1994 .

[14]  P. Wolfe.  Convergence Conditions for Ascent Methods. II , 1969 .