An adaptive augmented Lagrangian method for large-scale constrained optimization

We propose an augmented Lagrangian algorithm for solving large-scale constrained optimization problems. The novel feature of the algorithm is an adaptive update for the penalty parameter motivated by recently proposed techniques for exact penalty methods. This adaptive updating scheme greatly improves the overall performance of the algorithm without sacrificing the strengths of the core augmented Lagrangian framework, such as its ability to be implemented matrix-free. This is important as this feature of augmented Lagrangian methods is responsible for renewed interests in employing such methods for solving large-scale problems. We provide convergence results from remote starting points and illustrate by a set of numerical experiments that our method outperforms traditional augmented Lagrangian methods in terms of critical performance measures.

[1]  Jorge Nocedal,et al.  Steering exact penalty methods for nonlinear programming , 2008, Optim. Methods Softw..

[2]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[3]  M. Solodov,et al.  Local convergence of exact and inexact augmented Lagrangian methods under the second-order sufficiency condition , 2010 .

[4]  P. Boggs,et al.  A family of descent functions for constrained optimization , 1984 .

[5]  Mahmoud El-Alem.  A Global Convergence Theory for Dennis, El-Alem, and Maciel's Class of Trust-Region Algorithms for Constrained Optimization without Assuming Regularity , 1999, SIAM J. Optim..

[6]  P. Gill,et al.  Some theoretical properties of an augmented lagrangian merit function , 1986 .

[7]  Mikhail V. Solodov,et al.  Local Convergence of Exact and Inexact Augmented Lagrangian Methods under the Second-Order Sufficient Optimality Condition , 2012, SIAM J. Optim..

[8]  Daniel P. Robinson,et al.  A Globally Convergent Stabilized SQP Method , 2013, SIAM J. Optim..

[9]  Klaus Schittkowski,et al.  More test examples for nonlinear programming codes , 1981 .

[10]  Nicholas I. M. Gould,et al.  Lancelot: A FORTRAN Package for Large-Scale Nonlinear Optimization (Release A) , 1992 .

[11]  Alexey F. Izmailov,et al.  On attraction of linearly constrained Lagrangian methods and of stabilized and quasi-Newton SQP methods to critical multipliers , 2011, Math. Program..

[12]  R. Fletcher,et al.  A Class of Methods for Nonlinear Programming II Computational Experience , 1970 .

[13]  Jorge Nocedal,et al.  A line search exact penalty method using steering rules , 2010, Mathematical Programming.

[14]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[15]  Stephen J. Wright.  Superlinear Convergence of a Stabilized SQP Method to a Degenerate Solution , 1998, Comput. Optim. Appl..

[16]  Alexey F. Izmailov,et al.  Stabilized SQP revisited , 2012, Math. Program..

[17]  M. Kocvara.  A Generalized Augmented Lagrangian Method for Semidefinite Programming , 2003 .

[18]  R. Glowinski,et al.  Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de Dirichlet non linéaires , 1975 .

[19]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[20]  M. J. D. Powell,et al.  A method for nonlinear constraints in minimization problems , 1969 .

[21]  Nicholas I. M. Gould,et al.  CUTEr and SifDec: A constrained and unconstrained testing environment, revisited , 2003, TOMS.

[22]  Shiqian Ma,et al.  An alternating direction method for total variation denoising , 2011, Optim. Methods Softw..

[23]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[24]  L. Grippo,et al.  A New Class of Augmented Lagrangians in Nonlinear Programming , 1979 .

[25]  José Mario Martínez,et al.  Augmented Lagrangian method with nonmonotone penalty parameters for constrained optimization , 2011, Computational Optimization and Applications.

[26]  Stephen J. Wright,et al.  Numerical Behavior of a Stabilized SQP Method for Degenerate NLP Problems , 2002, COCOS.

[27]  Klaus Schittkowski,et al.  Test examples for nonlinear programming codes , 1980 .

[28]  Jorge J. Moré,et al.  Trust regions and projected gradients , 1988 .

[29]  John E. Dennis,et al.  A Global Convergence Theory for General Trust-Region-Based Algorithms for Equality Constrained Optimization , 1997, SIAM J. Optim..

[30]  Philippe L. Toint.  Nonlinear stepsize control, trust regions and regularizations for unconstrained optimization , 2013, Optim. Methods Softw..

[31]  Michael A. Saunders,et al.  SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization , 2002, SIAM J. Optim..

[32]  M. Hestenes.  Multiplier and gradient methods , 1969 .

[33]  Mikhail V. Solodov,et al.  Stabilized sequential quadratic programming for optimization and a stabilized Newton-type method for variational problems , 2010, Math. Program..

[34]  V. Torczon,et al.  A GLOBALLY CONVERGENT AUGMENTED LAGRANGIAN ALGORITHM FOR OPTIMIZATION WITH GENERAL CONSTRAINTS AND SIMPLE BOUNDS , 2002 .

[35]  Jorge J. Moré,et al.  Benchmarking optimization software with COPS. , 2001 .

[36]  R. Dembo,et al.  INEXACT NEWTON METHODS , 1982 .

[37]  José Mario Martínez,et al.  Augmented Lagrangian methods under the constant positive linear dependence constraint qualification , 2007, Math. Program..

[38]  Kenneth R. Davidson,et al.  Real Analysis and Applications , 2010 .

[39]  A. Sartenaer,et al.  Automatic decrease of the penalty parameter in exact penalty function methods , 1995 .

[40]  Junfeng Yang,et al.  A Fast Alternating Direction Method for TVL1-L2 Signal Reconstruction From Partial Fourier Data , 2010, IEEE Journal of Selected Topics in Signal Processing.