Curvilinear path and trust region in unconstrained optimization: a convergence analysis

In this paper we propose a general algorithm for solving unconstrained optimization problems. The basic step of the algorithm consists in finding a “good” successor point to the current iterate by choosing it along a curvilinear path and within a trust region. This scheme is due to Powell and it has been applied by Sorensen to a particular type of path. We give a series of properties that an arbitrary path should satisfy in order to achieve global convergence and fast asymptotic convergence. We review various paths that have been proposed in the literature and study the extent to which they satisfy our properties.

[1]  M. J. D. Powell,et al.  On the global convergence of trust region algorithms for unconstrained minimization , 1984, Math. Program..

[2]  Trond Steihaug,et al.  Truncated-newtono algorithms for large-scale unconstrained optimization , 1983, Math. Program..

[3]  M. J. D. Powell,et al.  Nonlinear optimization, 1981 , 1982 .

[4]  D. Jacobson,et al.  A Newton-type curvilinear search method for optimization , 1976 .

[5]  Donald Goldfarb,et al.  Curvilinear path steplength algorithms for minimization which use directions of negative curvature , 1980, Math. Program..

[6]  Jean-Philippe Vial,et al.  Unconstrained Optimization by Approximation of the Gradient Path , 1977, Math. Oper. Res..

[7]  J. Dennis,et al.  Two new unconstrained optimization algorithms which use function and gradient values , 1979 .

[8]  Philip Rabinowitz,et al.  Numerical methods for nonlinear algebraic equations , 1970 .

[9]  T. Steihaug.  The Conjugate Gradient Method and Trust Regions in Large Scale Optimization , 1983 .

[10]  M. Powell.  CONVERGENCE PROPERTIES OF A CLASS OF MINIMIZATION ALGORITHMS , 1975 .

[11]  S. W. Thomas.  Sequential estimation techniques for quasi-newton algorithms. , 1975 .

[12]  R. Fletcher,et al.  A modified Newton method for minimization , 1977 .

[13]  D. Sorensen.  Newton's method with a model trust region modification , 1982 .

[14]  Alfred Auslender,et al.  Penalty methods for computing points that satisfy second order necessary conditions , 1979, Math. Program..

[15]  Richard H. Byrd,et al.  A Family of Trust Region Based Algorithms for Unconstrained Minimization with Strong Global Convergence Properties. , 1985 .

[16]  S. Goldfeld,et al.  Maximization by Quadratic Hill-Climbing , 1966 .

[17]  Israel Zang,et al.  A new arc algorithm for unconstrained optimization , 1978, Math. Program..

[18]  W. Murray.  Numerical Methods for Unconstrained Optimization , 1975 .

[19]  Lamberto Cesari,et al.  Optimization-Theory And Applications , 1983 .

[20]  D. Marquardt.  An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[21]  Jorge J. Moré,et al.  Computing a Trust Region Step , 1983 .

[22]  Kenneth Levenberg.  A METHOD FOR THE SOLUTION OF CERTAIN NON – LINEAR PROBLEMS IN LEAST SQUARES , 1944 .

[23]  Philip E. Gill,et al.  Newton-type methods for unconstrained and linearly constrained optimization , 1974, Math. Program..

[24]  J. Vial,et al.  A restricted trust region algorithm for unconstrained optimization , 1985 .

[25]  Philippe L. Toint,et al.  Towards an efficient sparsity exploiting newton method for minimization , 1981 .

[26]  Danny C. Sorensen,et al.  On the use of directions of negative curvature in a modified newton method , 1979, Math. Program..