Line search algorithms with guaranteed sufficient decrease

The development of software for minimization problems is often based on a line search method. We consider line search methods that satisfy sufficient decrease and curvature conditions, and formulate the problem of determining a point that satisfies these two conditions in terms of finding a point in a set T(μ). We describe a search algorithm for this problem that produces a sequence of iterates that converge to a point in T(μ) and that, except for pathological cases, terminates in a finite number of steps. Numerical results for an implementation of the search algorithm on a set of test functions show that the algorithm terminates within a small number of iterations.

[1]  David F. Shanno,et al.  Algorithm 500: Minimization of Unconstrained Multivariate Functions [E4] , 1976, TOMS.

[2]  David F. Shanno,et al.  Remark on “Algorithm 500: Minimization of Unconstrained Multivariate Functions [E4]” , 1980, TOMS.

[3]  Claude Lemaréchal,et al.  A view of line-searches , 1981 .

[4]  J. J. Moré,et al.  Newton's Method , 1982 .

[5]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[6]  M. Al-Baali.  Descent Property and Global Convergence of the Fletcher—Reeves Method with Inexact Line Search , 1985 .

[7]  R. Fletcher,et al.  An efficient line search for nonlinear least squares , 1986 .

[8]  J. Nocedal,et al.  Global Convergence of a Class of Quasi-newton Methods on Convex Problems, Siam Some Global Convergence Properties of a Variable Metric Algorithm for Minimization without Exact Line Searches, Nonlinear Programming, Edited , 1996 .

[9]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[10]  D. O’Leary.  Robust regression computation computation using iteratively reweighted least squares , 1990 .

[11]  Jorge Nocedal,et al.  Global Convergence Properties of Conjugate Gradient Methods for Optimization , 1992, SIAM J. Optim..

[12]  Tamar Schlick,et al.  TNPACK—a truncated Newton minimization package for large-scale problems: II. Implementation examples , 1992, TOMS.

[13]  Tamar Schlick,et al.  TNPACK—A truncated Newton minimization package for large-scale problems: I. Algorithm and usage , 1992, TOMS.

[14]  W. W. Hager~f.  A Derivative-based Bracketing Scheme for Univariate Minimization and the Conjugate Gradient Method , 2022 .