On the convergence of interior-reflective Newton methods for nonlinear minimization subject to bounds

We consider a new algorithm, an interior-reflective Newton approach, for the problem of minimizing a smooth nonlinear function of many variables, subject to upper and/or lower bounds on some of the variables. This approach generatesstrictly feasible iterates by using a new affine scaling transformation and following piecewise linear paths (reflection paths). The interior-reflective approach does not require identification of an “activity set”. In this paper we establish that the interior-reflective Newton approach is globally and quadratically convergent. Moreover, we develop a specific example of interior-reflective Newton methods which can be used for large-scale and sparse problems.

[1]  A. Goldstein.  On Steepest Descent , 1965 .

[2]  L. Armijo.  Minimization of functions having Lipschitz continuous first partial derivatives. , 1966 .

[3]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[4]  R. Fletcher,et al.  Minimization of a Quadratic Function of Many Variables Subject only to Lower and Upper Bounds , 1974 .

[5]  D. O’Leary.  A generalized conjugate gradient algorithm for solving a class of quadratic programming problems , 1977 .

[6]  Donald Goldfarb,et al.  Curvilinear path steplength algorithms for minimization which use directions of negative curvature , 1980, Math. Program..

[7]  Richard H. Byrd,et al.  A Family of Trust Region Based Algorithms for Unconstrained Minimization with Strong Global Convergence Properties. , 1985 .

[8]  D. Sorensen.  Newton's method with a model trust region modification , 1982 .

[9]  Per Lötstedt.  Solving the minimal least squares problem subject to bounds on the variables , 1984 .

[10]  P. Toint,et al.  Testing a class of methods for solving minimization problems with simple bounds on the variables , 1988 .

[11]  Å. Björck.  A direct method for sparse least squares problems with lower and upper bounds , 1988 .

[12]  Richard H. Byrd,et al.  Approximate solution of the trust region problem by minimization over two-dimensional subspaces , 1988, Math. Program..

[13]  M. Todd,et al.  Recent Developments and New Directions in Linear Programming , 1988 .

[14]  J. J. Moré,et al.  Algorithms for bound constrained quadratic programming problems , 1989 .

[15]  Laurie A. Hulbert,et al.  A direct active set algorithm for large sparse quadratic programs with simple bounds , 1989, Math. Program..

[16]  Thomas F. Coleman,et al.  Computing a Trust Region Step for a Penalty Function , 1990, SIAM J. Sci. Comput..

[17]  T. Coleman,et al.  A Quadratically-Convergent Algorithm for the Linear Programming Problem with Lower and Upper Bounds , 1990 .

[18]  Jon W. Tolle,et al.  A class of methods for solving large, convex quadratic programs subject to box constraints , 1991, Math. Program..

[19]  Thomas F. Coleman,et al.  A globally and quadratically convergent affine scaling method for linearℓ1 problems , 1992, Math. Program..

[20]  T. Coleman,et al.  A global and quadratically convergent method for linear l ∞ problems , 1992 .

[21]  Laurie A. Hulbert,et al.  A Globally and Superlinearly Convergent Algorithm for Convex Quadratic Programs with Simple Bbounds , 1993, SIAM J. Optim..

[22]  Yuying Li,et al.  A Globally Convergent Method for lp Problems , 1991, SIAM J. Optim..

[23]  R. D. Murphy,et al.  Iterative solution of nonlinear equations , 1994 .

[24]  Thomas F. Coleman,et al.  A Reflective Newton Method for Minimizing a Quadratic Function Subject to Bounds on Some of the Variables , 1992, SIAM J. Optim..