A primal-dual trust region algorithm for nonlinear optimization

Abstract.This paper concerns general (nonconvex) nonlinear optimization when first and second derivatives of the objective and constraint functions are available. The proposed method is based on finding an approximate solution of a sequence of unconstrained subproblems parameterized by a scalar parameter. The objective function of each unconstrained subproblem is an augmented penalty-barrier function that involves both primal and dual variables. Each subproblem is solved using a second-derivative Newton-type method that employs a combined trust region and line search strategy to ensure global convergence. It is shown that the trust-region step can be computed by factorizing a sequence of systems with diagonally-modified primal-dual structure, where the inertia of these systems can be determined without recourse to a special factorization method. This has the benefit that off-the-shelf linear system software can be used at all times, allowing the straightforward extension to large-scale problems. Numerical results are given for problems in the COPS test collection.

[1]  M. Uddin.  Atomic Energy Research Establishment , 1948 .

[2]  M. Powell.  A method for nonlinear constraints in minimization problems , 1969 .

[3]  J. Bunch,et al.  Direct Methods for Solving Symmetric Indefinite Systems of Linear Equations , 1971 .

[4]  C. Reinsch.  Smoothing by spline functions. II , 1971 .

[5]  M. D. Hebden,et al.  An algorithm for minimization using exact second derivatives , 1973 .

[6]  J. J. Moré,et al.  A Characterization of Superlinear Convergence and its Application to Quasi-Newton Methods , 1973 .

[7]  J. Bunch,et al.  Some stable methods for calculating inertia and solving symmetric linear systems , 1977 .

[8]  S. W. Thomas.  Sequential estimation techniques for quasi-newton algorithms. , 1975 .

[9]  M. Powell.  CONVERGENCE PROPERTIES OF A CLASS OF MINIMIZATION ALGORITHMS , 1975 .

[10]  J. H. Wilkinson,et al.  AN ESTIMATE FOR THE CONDITION NUMBER OF A MATRIX , 1979 .

[11]  Danny C. Sorensen,et al.  On the use of directions of negative curvature in a modified newton method , 1979, Math. Program..

[12]  Klaus Schittkowski,et al.  Test examples for nonlinear programming codes , 1980 .

[13]  David M. author-Gay.  Computing Optimal Locally Constrained Steps , 1981 .

[14]  Klaus Schittkowski,et al.  More test examples for nonlinear programming codes , 1981 .

[15]  Philip E. Gill,et al.  Practical optimization , 1981 .

[16]  D. Sorensen.  Newton's method with a model trust region modification , 1982 .

[17]  Jorge J. Moré,et al.  Computing a Trust Region Step , 1983 .

[18]  N. Gould.  On the Accurate Determination of Search Directions for Simple Differentiable Penalty Functions , 1986 .

[19]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[20]  Nicholas J. Higham,et al.  FORTRAN codes for estimating the one-norm of a real or complex matrix, with applications to condition estimation , 1988, TOMS.

[21]  N. Higham.  Analysis of the Cholesky Decomposition of a Semi-definite Matrix , 1990 .

[22]  E. Omojokun.  Trust region algorithms for optimization with nonlinear equality and inequality constraints , 1990 .

[23]  P. Gill,et al.  Solving Reduced KKT Systems in Barrier Methods for Linear and Quadratic Programming , 1991 .

[24]  D. Ponceleón.  Barrier methods for large-scale quadratic programming , 1991 .

[25]  Margaret H. Wright,et al.  Interior methods for constrained optimization , 1992, Acta Numerica.

[26]  Yin Zhang,et al.  On the Superlinear and Quadratic Convergence of Primal-Dual Interior Point Linear Programming Algorithms , 1992, SIAM J. Optim..

[27]  Sanjay Mehrotra,et al.  Solving symmetric indefinite systems in an interior-point method for linear programming , 1993, Math. Program..

[28]  Yin Zhang,et al.  A Superlinearly Convergent Polynomial Primal-Dual Interior-Point Algorithm for Linear Programming , 1993, SIAM J. Optim..

[29]  W. Murray,et al.  Newton methods for large-scale linear equality-constrained minimization , 1993 .

[30]  Brian W. Kernighan,et al.  AMPL: A Modeling Language for Mathematical Programming , 1993 .

[31]  Yin Zhang,et al.  On the Superlinear Convergence of Interior-Point Algorithms for a General Class of Problems , 1993, SIAM J. Optim..

[32]  Anders Forsgren,et al.  Computing Modified Newton Directions Using a Partial Cholesky Factorization , 1995, SIAM J. Sci. Comput..

[33]  Stephen J. Wright.  Stability of Linear Equations Solvers in Interior-Point Methods , 1995, SIAM J. Matrix Anal. Appl..

[34]  Nicholas I. M. Gould,et al.  CUTE: constrained and unconstrained testing environment , 1995, TOMS.

[35]  L. N. Vicente,et al.  Trust-Region Interior-Point SQP Algorithms for a Class of Nonlinear Programming Problems , 1998 .

[36]  T. Tsuchiya,et al.  On the formulation and theory of the Newton interior-point method for nonlinear programming , 1996 .

[37]  Joseph R. Shinnerl,et al.  Stability of Symmetric Ill-Conditioned Systems Arising in Interior Methods for Constrained Optimization , 1996, SIAM J. Matrix Anal. Appl..

[38]  Stephen J. Wright.  Primal-Dual Interior-Point Methods , 1997, Other Titles in Applied Mathematics.

[39]  Michael L. Overton,et al.  A Primal-dual Interior Method for Nonconvex Nonlinear Programming , 1998 .

[40]  Anders Forsgren,et al.  Primal-Dual Interior Methods for Nonconvex Nonlinear Programming , 1998, SIAM J. Optim..

[41]  Margaret H. Wright,et al.  Ill-Conditioning and Computational Error in Interior Methods for Nonlinear Programming , 1998, SIAM J. Optim..

[42]  Robert J. Vanderbei,et al.  An Interior-Point Algorithm for Nonconvex Nonlinear Programming , 1999, Comput. Optim. Appl..

[43]  P. Gill,et al.  Combination trust-region line-search methods for unconstrained optimization , 1999 .

[44]  Jorge Nocedal,et al.  An Interior Point Algorithm for Large-Scale Nonlinear Programming , 1999, SIAM J. Optim..

[45]  Jorge J. Moré,et al.  COPS: Large-scale nonlinearly constrained optimization problems , 2000 .

[46]  Hande Y. Benson,et al.  INTERIOR-POINT METHODS FOR NONCONVEX NONLINEAR PROGRAMMING: JAMMING AND COMPARATIVE NUMERICAL TESTING , 2000 .

[47]  Robert J. Vanderbei,et al.  Interior-point methods for nonconvex nonlinear programming: orderings and higher-order methods , 2000, Math. Program..

[48]  Jorge Nocedal,et al.  A trust region method based on interior point techniques for nonlinear programming , 2000, Math. Program..

[49]  P. Toint,et al.  A primal-dual algorithm for minimizing a non-convex function subject to bound and linear equality constraints , 2000 .

[50]  Nicholas I. M. Gould,et al.  A primal-dual trust-region algorithm for non-convex nonlinear programming , 2000, Math. Program..

[51]  Nicholas I. M. Gould,et al.  Superlinear Convergence of Primal-Dual Interior Point Algorithms for Nonlinear Programming , 2000, SIAM J. Optim..

[52]  Todd Munson,et al.  Benchmarking optimization software with COPS. , 2001 .

[53]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[54]  Jorge J. Moré,et al.  Benchmarking optimization software with COPS. , 2001 .

[55]  R. Tapia,et al.  On the Global Convergence of a Modified Augmented Lagrangian Linesearch Interior-Point Newton Method for Nonlinear Programming , 2002 .

[56]  Robert J. Vanderbei,et al.  Interior-Point Methods for Nonconvex Nonlinear Programming: Filter Methods and Merit Functions , 2002, Comput. Optim. Appl..

[57]  Anders Forsgren,et al.  Interior Methods for Nonlinear Optimization , 2002, SIAM Rev..

[58]  Sven Leyffer,et al.  Nonlinear programming without a penalty function , 2002, Math. Program..

[59]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[60]  A. Forsgren.  Inertia-controlling factorizations for optimization algorithms , 2002 .

[61]  Stephen J. Wright,et al.  Properties of the Log-Barrier Function on Degenerate Nonlinear Programs , 2002, Math. Oper. Res..

[62]  ArgáezM.,et al.  On the global convergence of a modified augmented Lagrangian linesearch interior-point Newton method for nonlinear programming , 2002 .

[63]  Javier M. Moguerza,et al.  An augmented Lagrangian interior-point method using directions of negative curvature , 2003, Math. Program..

[64]  Stefan Ulbrich,et al.  A globally convergent primal-dual interior-point filter method for nonlinear programming , 2004, Math. Program..

[65]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.