Solving the Trust-Region Subproblem using the Lanczos Method

The approximate minimization of a quadratic function within an ellipsoidal trust region is an important subproblem for many nonlinear programming methods. When the number of variables is large, the most widely used strategy is to trace the path of conjugate gradient iterates either to convergence or until it reaches the trust-region boundary. In this paper, we investigate ways of continuing the process once the boundary has been encountered. The key is to observe that the trust-region problem within the currently generated Krylov subspace has a very special structure which enables it to be solved very efficiently. We compare the new strategy with existing methods. The resulting software package is available as HSL_VF05 within the Harwell Subroutine Library.

[1]  S. Goldfeld,et al.  Maximization by Quadratic Hill-Climbing , 1966 .

[2]  M. Powell.  A New Algorithm for Unconstrained Optimization , 1970 .

[3]  M. D. Hebden,et al.  An algorithm for minimization using exact second derivatives , 1973 .

[4]  M. Powell.  CONVERGENCE PROPERTIES OF A CLASS OF MINIMIZATION ALGORITHMS , 1975 .

[5]  J. Dennis,et al.  Two new unconstrained optimization algorithms which use function and gradient values , 1979 .

[6]  B. Parlett.  The Symmetric Eigenvalue Problem , 1981 .

[7]  Philippe L. Toint,et al.  Towards an efficient sparsity exploiting newton method for minimization , 1981 .

[8]  J. Reid,et al.  Tracking the Progress of the Lanczos Algorithm for Large Symmetric Eigenproblems , 1981 .

[9]  David M. author-Gay.  Computing Optimal Locally Constrained Steps , 1981 .

[10]  Richard H. Byrd,et al.  A Family of Trust Region Based Algorithms for Unconstrained Minimization with Strong Global Convergence Properties. , 1985 .

[11]  T. Steihaug.  The Conjugate Gradient Method and Trust Regions in Large Scale Optimization , 1983 .

[12]  Jorge J. Moré,et al.  Computing a Trust Region Step , 1983 .

[13]  S. Nash.  Newton-Type Minimization via the Lanczos Method , 1984 .

[14]  Elizabeth Eskow,et al.  A New Modified Cholesky Factorization , 1990, SIAM J. Sci. Comput..

[15]  P. Toint,et al.  Lancelot: A FORTRAN Package for Large-Scale Nonlinear Optimization (Release A) , 1992 .

[16]  R. Vanderbei,et al.  Max-min eigenvalue problems, primal-dual Interior point algorithms, and Trust region subproblemst , 1995 .

[17]  Nicholas I. M. Gould,et al.  CUTE: constrained and unconstrained testing environment , 1995, TOMS.

[18]  Stefano Lucidi,et al.  Numerical Experiences with New Truncated Newton Methods in Large Scale Unconstrained Optimization , 1997, Comput. Optim. Appl..

[19]  Danny C. Sorensen,et al.  Minimization of a Large-Scale Quadratic FunctionSubject to a Spherical Constraint , 1997, SIAM J. Optim..

[20]  Franz Rendl,et al.  A semidefinite framework for trust region subproblems with applications to large scale minimization , 1997, Math. Program..

[21]  Danny C. Sorensen,et al.  A New Matrix-Free Algorithm for the Large-Scale Trust-Region Subproblem , 2000, SIAM J. Optim..