AN INTERIOR-POINT SUBSPACE MINIMIZATION METHOD FOR THE TRUST-REGION STEP

Abstract. We consider methods for large-scale unconstrained minimization based on finding an approximate minimizer of a quadratic function subject to a two-norm trust-region inequality constraint. The Steihaug-Toint method uses the conjugate-gradient algorithm to minimize the quadratic over a sequence of expanding subspaces until the iterates either converge to an interior point or cross the constraint boundary. Recent extensions of the Steihaug-Toint method allow the accuracy of the trust-region step to be specified, thereby allowing the overall cost of computing the problem functions to be balanced against the cost of computing the trust-region steps. However, if a preconditioner is used with the conjugate-gradient algorithm, the Steihaug-Toint method requires the trust-region norm to be defined in terms of the preconditioning matrix. If a different preconditioner is used for each subproblem, the shape of the trust-region can change substantially from one subproblem to the next, which invalidates many of the assumptions on which standard methods for adjusting the trust-region radius are based. In this paper we propose a method that allows the trust-region norm to be defined independently of the preconditioner. The method solves the inequality constrained trust-region subproblem over a sequence of evolving low-dimensional subspaces. Each subspace includes an accelerator direction obtained from a Newton method applied to an primal-dual interior method. A crucial property of this direction is that it can be computed by applying the preconditioned conjugate-gradient method to a positive-definite system in both the primal and dual variables of the trust-region subproblem.

[1]  William W. Hager,et al.  Global convergence of SSM for minimizing a quadratic over a sphere , 2004, Math. Comput..

[2]  Philip E. Gill,et al.  Reduced-Hessian Quasi-Newton Methods for Unconstrained Optimization , 2001, SIAM J. Optim..

[3]  D. Sorensen.  Newton's method with a model trust region modification , 1982 .

[4]  Jorge J. Moré,et al.  Computing a Trust Region Step , 1983 .

[5]  Franz Rendl,et al.  A semidefinite framework for trust region subproblems with applications to large scale minimization , 1997, Math. Program..

[6]  David M. author-Gay.  Computing Optimal Locally Constrained Steps , 1981 .

[7]  Danny C. Sorensen,et al.  Minimization of a Large-Scale Quadratic FunctionSubject to a Spherical Constraint , 1997, SIAM J. Optim..

[8]  Dirk Siegel,et al.  Updating of conjugate direction matrices using members of Broyden's family , 1993, Math. Program..

[9]  Chih-Jen Lin,et al.  Incomplete Cholesky Factorizations with Limited Memory , 1999, SIAM J. Sci. Comput..

[10]  Philip E. Gill,et al.  Iterative Methods for Finding a Trust-region Step , 2009, SIAM J. Optim..

[11]  Nicholas I. M. Gould,et al.  Solving the Trust-Region Subproblem using the Lanczos Method , 1999, SIAM J. Optim..

[12]  Danny C. Sorensen,et al.  A Trust-Region Approach to the Regularization of Large-Scale Discrete Forms of Ill-Posed Problems , 2001, SIAM J. Sci. Comput..

[13]  Nicholas I. M. Gould,et al.  CUTEr and SifDec: A constrained and unconstrained testing environment, revisited , 2003, TOMS.

[14]  Philippe L. Toint,et al.  Towards an efficient sparsity exploiting newton method for minimization , 1981 .

[15]  Anders Forsgren,et al.  Primal-Dual Interior Methods for Nonconvex Nonlinear Programming , 1998, SIAM J. Optim..

[16]  M. Powell.  A New Algorithm for Unconstrained Optimization , 1970 .

[17]  Dirk Siegel,et al.  Modifying the BFGS update by a new column scaling technique , 1994, Math. Program..

[18]  Roman A. Polyak,et al.  Modified barrier functions (theory and methods) , 1992, Math. Program..

[19]  P. Gill,et al.  Combination trust-region line-search methods for unconstrained optimization , 1999 .

[20]  Nicholas I. M. Gould,et al.  CUTE: constrained and unconstrained testing environment , 1995, TOMS.

[21]  Philip E. Gill,et al.  Limited-Memory Reduced-Hessian Methods for Large-Scale Unconstrained Optimization , 2003, SIAM J. Optim..

[22]  William W. Hager,et al.  Minimizing a Quadratic Over a Sphere , 2001, SIAM J. Optim..

[23]  Danny C. Sorensen,et al.  A New Matrix-Free Algorithm for the Large-Scale Trust-Region Subproblem , 2000, SIAM J. Optim..

[24]  John L. Nazareth.  The method of successive affine reduction for nonlinear minimization , 1986, Math. Program..

[25]  M. Powell.  CONVERGENCE PROPERTIES OF A CLASS OF MINIMIZATION ALGORITHMS , 1975 .

[26]  T. Steihaug.  The Conjugate Gradient Method and Trust Regions in Large Scale Optimization , 1983 .

[27]  採編典藏組.  Society for Industrial and Applied Mathematics(SIAM) , 2008 .

[28]  R. Dembo,et al.  INEXACT NEWTON METHODS , 1982 .

[29]  Anders Forsgren,et al.  Interior Methods for Nonlinear Optimization , 2002, SIAM Rev..

[30]  Jorge J. Moré,et al.  Benchmarking optimization software with COPS. , 2001 .

[31]  Nicholas I. M. Gould,et al.  A globally convergent Lagrangian barrier algorithm for optimization with general inequality constraints and simple bounds , 1997, Math. Comput..