A Global Convergence Theory for General Trust-Region-Based Algorithms for Equality Constrained Optimization

This work presents a global convergence theory for a broad class of trust-region algorithms for the smooth nonlinear programming problem with equality constraints. The main result generalizes Powell's 1975 result for unconstrained trust-region algorithms. The trial step is characterized by very mild conditions on its normal and tangential components. The normal component need not be computed accurately. The theory requires a quasi-normal component to satisfy a fraction of Cauchy decrease condition on the quadratic model of the linearized constraints. The tangential component then must satisfy a fraction of Cauchy decrease condition on a quadratic model of the Lagrangian function in the translated tangent space of the constraints determined by the quasi-normal component. Estimates of the Lagrange multipliers and the Hessians are assumed only to be bounded. The other main characteristic of this class of algorithms is that the step is evaluated by using the augmented Lagrangian as a merit function with the penalty parameter updated using the El-Alem scheme. The properties of the step and the way that the penalty parameter is chosen are sufficient to establish global convergence. As an example, an algorithm is presented that can be viewed as a generalization of the Steihaug--Toint dogleg algorithm for the unconstrained case. It is based on a quadratic programming algorithm that uses a step in a quasi-normal direction to the tangent space of the constraints and then takes feasible conjugate reduced-gradient steps to solve the reduced quadratic program. This algorithm should cope quite well with large problems for which effective preconditioners are known.

[1]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[2]  R. Tapia,et al.  A Global Convergence Theory for Arbitrary Norm Trust-Region Methods for Nonlinear Equations , 1995 .

[3]  Jorge J. Moré,et al.  Recent Developments in Algorithms and Software for Trust Region Methods , 1982, ISMP.

[4]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[5]  M. Powell.  CONVERGENCE PROPERTIES OF A CLASS OF MINIMIZATION ALGORITHMS , 1975 .

[6]  James E. Dennis,et al.  A Comparison of Nonlinear Programming Approaches to an Elliptic Inverse Problem and a New Domain Dec , 1994 .

[7]  K. Schittkowski.  On the Convergence of a Sequential Quadratic Programming Method with an Augmented Lagrangian Line Search Functions. , 1982 .

[8]  D. Sorensen.  Newton's method with a model trust region modification , 1982 .

[9]  M. El-Alem.  A global convergence theory for the Celis-Dennis-Tapia trust-region algorithm for constrained optimization , 1991 .

[10]  Leon S. Lasdon,et al.  An Improved Successive Linear Programming Algorithm , 1985 .

[11]  Richard H. Byrd,et al.  A Family of Trust Region Based Algorithms for Unconstrained Minimization with Strong Global Convergence Properties. , 1985 .

[12]  R. A. Tapia,et al.  QUASI-NEWTON METHODS FOR EQUALITY CONSTRAINED OPTIMIZATION: EQUIVALENCE OF EXISTING METHODS AND A NEW IMPLEMENTATION , 1978 .

[13]  Mahmoud El-Alem,et al.  A Robust Trust-Region Algorithm with a Nonmonotonic Penalty Parameter Scheme for Constrained Optimization , 1995, SIAM J. Optim..

[14]  P. Gill,et al.  Some theoretical properties of an augmented lagrangian merit function , 1986 .

[15]  Roger Fletcher,et al.  An exact penalty function for nonlinear programming with inequalities , 1973, Math. Program..

[16]  A. Vardi.  A Trust Region Algorithm for Equality Constrained Minimization: Convergence Properties and Implementation , 1985 .

[17]  Jorge J. Moré,et al.  Computing a Trust Region Step , 1983 .

[18]  Richard H. Byrd,et al.  A Trust Region Algorithm for Nonlinearly Constrained Optimization , 1987 .

[19]  R. Tapia.  On secant updates for use in general constrained optimization , 1988 .

[20]  John E. Dennis,et al.  A global convergence theory for a general class of trust region algorithms for equality constrained optimization , 1993 .

[21]  S. Nash.  Preconditioning of Truncated-Newton Methods , 1985 .

[22]  Philippe L. Toint,et al.  Towards an efficient sparsity exploiting newton method for minimization , 1981 .

[23]  R. Carter.  Multi-Model Algorithms for Optimization , 1986 .

[24]  T. Steihaug.  The Conjugate Gradient Method and Trust Regions in Large Scale Optimization , 1983 .

[25]  M. R. Celis.  A TRUST REGION STRATEGY FOR NONLINEAR EQUALITY CONSTRAINED OPTIMIZATION (NONLINEAR PROGRAMMING, SEQUENTIAL QUADRATIC) , 1985 .

[26]  Ya-Xiang Yuan,et al.  A trust region algorithm for equality constrained optimization , 1990, Math. Program..

[27]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[28]  Andreas Griewank,et al.  ADIFOR - Generating Derivative Codes form Fortran Programs , 1992, Sci. Program..

[29]  J. Dennis,et al.  A robust trust region algorithm for nonlinear programming , 1990 .

[30]  Philip E. Gill,et al.  Practical optimization , 1981 .

[31]  E. J. Craig.  The N‐Step Iteration Procedures , 1955 .

[32]  J. Dennis,et al.  A global convergence theory for a class of trust region algorithms for constrained optimization , 1988 .