An augmented Lagrangian trust region method for equality constrained optimization

In this paper we propose an augmented Lagrangian trust region method for equality constrained optimization. Different from standard augmented Lagrangian methods which minimize the augmented Lagrangian function for fixed Lagrange multiplier and penalty parameter at each iteration, the proposed method tries to minimize its second-order approximation function. We propose a new strategy for adjusting the penalty parameter. With adaptive update of Lagrange multipliers, we prove the global convergence of the proposed method. Numerical results on test problems from the CUTEr collection are also reported.

[1]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[2]  R. Andreani,et al.  On the Relation between Constant Positive Linear Dependence Condition and Quasinormality Constraint Qualification , 2005 .

[3]  Richard H. Byrd,et al.  Approximate solution of the trust region problem by minimization over two-dimensional subspaces , 1988, Math. Program..

[4]  R. Fletcher.  NUMERICAL EXPERIMENTS WITH AN EXACT L1 PENALTY FUNCTION METHOD , 1981 .

[5]  Ya-Xiang Yuan,et al.  A trust region algorithm for equality constrained optimization , 1990, Math. Program..

[6]  Necdet Serhat Aybat,et al.  A First-Order Augmented Lagrangian Method for Compressed Sensing , 2010, SIAM J. Optim..

[7]  M. Hestenes.  Multiplier and gradient methods , 1969 .

[8]  Zdenek Dostál,et al.  Semi-monotonic inexact augmented Lagrangians for quadratic programing with equality constraints , 2005, Optim. Methods Softw..

[9]  Ya-Xiang Yuan,et al.  On the convergence of a new trust region algorithm , 1995 .

[10]  William W. Hager,et al.  Global convergence of SSM for minimizing a quadratic over a sphere , 2004, Math. Comput..

[11]  Ya-Xiang Yuan,et al.  A NEW TRUST-REGION ALGORITHM FOR NONLINEAR CONSTRAINED OPTIMIZATION * , 2009 .

[12]  Richard H. Byrd,et al.  A Trust Region Algorithm for Nonlinearly Constrained Optimization , 1987 .

[13]  José Mario Martínez,et al.  Augmented Lagrangian methods under the constant positive linear dependence constraint qualification , 2007, Math. Program..

[14]  Kim-Chuan Toh,et al.  A Newton-CG Augmented Lagrangian Method for Semidefinite Programming , 2010, SIAM J. Optim..

[15]  Nicholas I. M. Gould,et al.  Lancelot: A FORTRAN Package for Large-Scale Nonlinear Optimization (Release A) , 1992 .

[16]  Paulo J. S. Silva,et al.  A relaxed constant positive linear dependence constraint qualification and applications , 2011, Mathematical Programming.

[17]  José Mario Martínez,et al.  On Augmented Lagrangian Methods with General Lower-Level Constraints , 2007, SIAM J. Optim..

[18]  N. Maratos,et al.  Exact penalty function algorithms for finite dimensional and control optimization problems , 1978 .

[19]  P. Toint,et al.  A globally convergent augmented Lagrangian algorithm for optimization with general constraints and simple bounds , 1991 .

[20]  Jorge J. Moré,et al.  Computing a Trust Region Step , 1983 .

[21]  Jorge J. Moré,et al.  Benchmarking optimization software with performance profiles , 2001, Math. Program..

[22]  Yousef Saad,et al.  Hybrid Krylov Methods for Nonlinear Systems of Equations , 1990, SIAM J. Sci. Comput..

[23]  M. J. D. Powell,et al.  Algorithms for nonlinear constraints that use lagrangian functions , 1978, Math. Program..

[24]  M. J. D. Powell,et al.  A method for nonlinear constraints in minimization problems , 1969 .

[25]  Shih-Ping Han,et al.  Superlinearly convergent variable metric algorithms for general nonlinear programming problems , 1976, Math. Program..

[26]  D. Goldfarb,et al.  Solving low-rank matrix completion problems efficiently , 2009, 2009 47th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[27]  Zengxin Wei,et al.  On the Constant Positive Linear Dependence Condition and Its Application to SQP Methods , 1999, SIAM J. Optim..

[28]  Ya-Xiang Yuan,et al.  On the truncated conjugate gradient method , 2000, Math. Program..

[29]  R. Rockafellar.  The multiplier method of Hestenes and Powell applied to convex programming , 1973 .

[30]  William W. Hager,et al.  Minimizing a Quadratic Over a Sphere , 2001, SIAM J. Optim..

[31]  Zdenek Dostál,et al.  Augmented Lagrangians with Adaptive Precision Control for Quadratic Programming with Simple Bounds and Equality Constraints , 2002, SIAM J. Optim..

[32]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[33]  Hao Jiang,et al.  An adaptive augmented Lagrangian method for large-scale constrained optimization , 2014, Mathematical Programming.

[34]  Nicholas I. M. Gould,et al.  CUTEr and SifDec: A constrained and unconstrained testing environment, revisited , 2003, TOMS.