Newton's Method for Large Bound-Constrained Optimization Problems

We analyze a trust region version of Newton's method for bound-constrained problems. Our approach relies on the geometry of the feasible set, not on the particular representation in terms of constraints. The convergence theory holds for linearly constrained problems and yields global and superlinear convergence without assuming either strict complementarity or linear independence of the active constraints. We also show that the convergence theory leads to an efficient implementation for large bound-constrained problems.

[1]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[2]  Stephen M. Robinson,et al.  Strongly Regular Generalized Equations , 1980, Math. Oper. Res..

[3]  N. Munksgaard,et al.  Solving Sparse Symmetric Sets of Linear Equations by Preconditioned Conjugate Gradients , 1980, TOMS.

[4]  D. Bertsekas.  Projected Newton methods for optimization problems with simple constraints , 1981, 1981 20th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[5]  T. Steihaug.  The Conjugate Gradient Method and Trust Regions in Large Scale Optimization , 1983 .

[6]  Jorge J. Moré,et al.  Computing a Trust Region Step , 1983 .

[7]  D. Bertsekas,et al.  TWO-METRIC PROJECTION METHODS FOR CONSTRAINED OPTIMIZATION* , 1984 .

[8]  Paul H. Calamai,et al.  Projected gradient methods for linearly constrained problems , 1987, Math. Program..

[9]  J. Dunn.  On the convergence of projected gradient processes to singular critical points , 1987 .

[10]  P. Toint.  Global Convergence of a a of Trust-Region Methods for Nonconvex Minimization in Hilbert Space , 1988 .

[11]  P. Toint,et al.  Global convergence of a class of trust region algorithms for optimization with simple bounds , 1988 .

[12]  Jorge J. Moré,et al.  Trust regions and projected gradients , 1988 .

[13]  J. J. Moré,et al.  On the identification of active constraints , 1988 .

[14]  Gerardo Toraldo,et al.  Convergence properties of trust region methods for linear and convex constraints , 1990, Math. Program..

[15]  M. Lescrenier.  Convergence of trust region algorithms for optimization with bounds when strict complementarity does not hold , 1991 .

[16]  Gerardo Toraldo,et al.  On the Solution of Large Quadratic Programming Problems with Bound Constraints , 1991, SIAM J. Optim..

[17]  Guoliang Xue,et al.  The MINPACK-2 test problem collection , 1992 .

[18]  Nicholas I. M. Gould,et al.  Global Convergence of a Class of Trust Region Algorithms for Optimization Using Inexact Projections on Convex Constraints , 1993, SIAM J. Optim..

[19]  J. M. Martínez,et al.  A new trust region algorithm for bound constrained minimization , 1994 .

[20]  James V. Burke,et al.  Exposing Constraints , 1994, SIAM J. Optim..

[21]  Nicholas I. M. Gould,et al.  CUTE: constrained and unconstrained testing environment , 1995, TOMS.

[22]  Nicholas I. M. Gould,et al.  Numerical experiments with the LANCELOT package (release A) for large-scale nonlinear optimization , 1996, Math. Program..

[23]  L. N. Vicente,et al.  Trust-Region Interior-Point Algorithms for Minimization Problems with Simple Bounds , 1996 .

[24]  Jorge J. Moré,et al.  Global Methods for Nonlinear Complementarity Problems , 1994, Math. Oper. Res..

[25]  Thomas F. Coleman,et al.  An Interior Trust Region Approach for Nonlinear Minimization Subject to Bounds , 1993, SIAM J. Optim..

[26]  Anders Forsgren,et al.  Newton Methods For Large-Scale Linear Inequality-Constrained Minimization , 1997, SIAM J. Optim..

[27]  Jorge Nocedal,et al.  Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization , 1997, TOMS.

[28]  E. Polak,et al.  Family of Projected Descent Methods for Optimization Problems with Simple Bounds , 1997 .

[29]  T. Coleman,et al.  Combining Trust Region and Affine Scaling Linearly Constrained Nonconvex Minimization , 1997 .

[30]  Francisco Facchinei,et al.  An Active Set Newton Algorithm for Large-Scale Nonlinear Programs with Box Constraints , 1998, SIAM J. Optim..

[31]  Stefan Ulbrich,et al.  Superlinear and quadratic convergence of affine-scaling interior-point Newton methods for problems with simple bounds without strict complementarity assumption , 1999, Math. Program..

[32]  Chih-Jen Lin,et al.  Incomplete Cholesky Factorizations with Limited Memory , 1999, SIAM J. Sci. Comput..

[33]  Thomas F. Coleman,et al.  A trust region and affine scaling interior point method for nonconvex minimization with linear inequality constraints , 1997, Math. Program..