Approximate solution of system of equations arising in interior-point methods for bound-constrained optimization

The focus in this paper is interior-point methods for bound-constrained nonlinear optimization where the system of nonlinear equations that arise are solved with Newton's method. There is a trade-off between solving Newton systems directly, which give high quality solutions, and solving many approximate Newton systems which are computationally less expensive but give lower quality solutions. We propose partial and full approximate solutions to the Newton systems, which in general involves solving a reduced system of linear equations. The specific approximate solution and the size of the reduced system that needs to be solved at each iteration are determined by estimates of the active and inactive constraints at the solution. These sets are at each iteration estimated by a simple heuristic. In addition, we motivate and suggest two modified-Newton approaches which are based on an intermediate step that consists of the partial approximate solutions. The theoretical setting is introduced and asymptotic error bounds are given along with numerical results for bound-constrained convex quadratic optimization problems, both random and from the CUTEst test collection.

[1]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[2]  D. Bertsekas.  On the Goldstein-Levitin-Polyak gradient projection method , 1974, CDC 1974.

[3]  D. Bertsekas.  Projected Newton methods for optimization problems with simple constraints , 1981, 1981 20th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[4]  P. Toint,et al.  Global convergence of a class of trust region algorithms for optimization with simple bounds , 1988 .

[5]  Michael A. Saunders,et al.  Preconditioners for Indefinite Systems Arising in Optimization , 1992, SIAM J. Matrix Anal. Appl..

[6]  Thomas F. Coleman,et al.  On the convergence of interior-reflective Newton methods for nonlinear minimization subject to bounds , 1994, Math. Program..

[7]  Jorge Nocedal,et al.  A Limited Memory Algorithm for Bound Constrained Optimization , 1995, SIAM J. Sci. Comput..

[8]  Stephen J. Wright.  Stability of Linear Equations Solvers in Interior-Point Methods , 1995, SIAM J. Matrix Anal. Appl..

[9]  Joseph R. Shinnerl,et al.  Stability of Symmetric Ill-Conditioned Systems Arising in Interior Methods for Constrained Optimization , 1996, SIAM J. Matrix Anal. Appl..

[10]  Jorge Nocedal,et al.  Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization , 1997, TOMS.

[11]  E. Polak,et al.  Family of Projected Descent Methods for Optimization Problems with Simple Bounds , 1997 .

[12]  Jorge Nocedal,et al.  On the Local Behavior of an Interior Point Method for Nonlinear Programming , 1997 .

[13]  Anders Forsgren,et al.  Primal-Dual Interior Methods for Nonconvex Nonlinear Programming , 1998, SIAM J. Optim..

[14]  Francisco Facchinei,et al.  An Active Set Newton Algorithm for Large-Scale Nonlinear Programs with Box Constraints , 1998, SIAM J. Optim..

[15]  Margaret H. Wright,et al.  Ill-Conditioning and Computational Error in Interior Methods for Nonlinear Programming , 1998, SIAM J. Optim..

[16]  Robert J. Vanderbei,et al.  An Interior-Point Algorithm for Nonconvex Nonlinear Programming , 1999, Comput. Optim. Appl..

[17]  J. Pang,et al.  Computational optimization : a tribute to Olvi Mangasarian , 1999 .

[18]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[19]  Stefan Ulbrich,et al.  Superlinear and quadratic convergence of affine-scaling interior-point Newton methods for problems with simple bounds without strict complementarity assumption , 1999, Math. Program..

[20]  Chih-Jen Lin,et al.  Newton's Method for Large Bound-Constrained Optimization Problems , 1999, SIAM J. Optim..

[21]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[22]  Stephen J. Wright.  Effects of Finite-Precision Arithmetic on Interior-Point Methods for Nonlinear Programming , 2001, SIAM J. Optim..

[23]  Anders Forsgren,et al.  Interior Methods for Nonlinear Optimization , 2002, SIAM Rev..

[24]  Nicholas I. M. Gould,et al.  Numerical methods for large-scale nonlinear optimization , 2005, Acta Numerica.

[25]  Lorenz T. Biegler,et al.  On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming , 2006, Math. Program..

[26]  Jorge Nocedal,et al.  An interior algorithm for nonlinear optimization that combines line search and trust region steps , 2006, Math. Program..

[27]  Christian Kanzow,et al.  On Affine-Scaling Interior-Point Newton Methods for Nonlinear Minimization with Bound Constraints , 2006, Comput. Optim. Appl..

[28]  William W. Hager,et al.  A New Active Set Algorithm for Box Constrained Optimization , 2006, SIAM J. Optim..

[29]  Anders Forsgren,et al.  Iterative Solution of Augmented Systems Arising in Interior Methods , 2007, SIAM J. Optim..

[30]  S. Nash,et al.  Linear and Nonlinear Optimization , 2008 .

[31]  Inderjit S. Dhillon,et al.  Tackling Box-Constrained Optimization via a New Projected Quasi-Newton Approach , 2010, SIAM J. Sci. Comput..

[32]  Nicholas I. M. Gould,et al.  CUTEst: a Constrained and Unconstrained Testing Environment with safe threads for mathematical optimization , 2013, Computational Optimization and Applications.

[33]  Valeria Simoncini,et al.  Stability and Accuracy of Inexact Interior Point Methods for Convex Quadratic Programming , 2017, J. Optim. Theory Appl..

[34]  Jacek Gondzio,et al.  Quasi-Newton approaches to interior point methods for quadratic problems , 2018, Computational Optimization and Applications.