Decomposition Methods Based on Augmented Lagrangians: A Survey

In this chapter, we provide a non-exhaustive account of decomposition algorithms for solving structured large scale convex and non-convex optimization problemswith major emphasis on several splitting approaches based on the classical or modified augmented Lagrangian functions. This study covers last 40 years of research on theoretical properties of augmented Lagrangians.

[1]  R. Fletcher,et al.  Some numerical experience with a globally convergent algorithm for nonlinearly constrained optimization , 1980 .

[2]  Robert R. Meyer,et al.  A variable-penalty alternating directions method for convex optimization , 1998, Math. Program..

[3]  Abdelouahed Hamdi,et al.  Decomposition for structured convex programs with smooth multiplier methods , 2005, Appl. Math. Comput..

[4]  D. Bertsekas.  Convexification procedures and decomposition methods for nonconvex optimization problems , 1979 .

[5]  G. Stephanopoulos,et al.  The use of Hestenes' method of multipliers to resolve dual gaps in engineering system optimization , 1975 .

[6]  M. Hestenes.  Multiplier and gradient methods , 1969 .

[7]  R. Fletcher,et al.  A Class of Methods for Nonlinear Programming II Computational Experience , 1970 .

[8]  Y. Censor,et al.  Proximal minimization algorithm withD-functions , 1992 .

[9]  Bingsheng He,et al.  A new inexact alternating directions method for monotone variational inequalities , 2002, Math. Program..

[10]  Jonathan E. Spingarn,et al.  Applications of the method of partial inverses to convex programming: Decomposition , 1985, Math. Program..

[11]  Abdelouahed Hamdi,et al.  Two-level primal-dual proximal decomposition technique to solve large scale optimization problems , 2005, Appl. Math. Comput..

[12]  Paul Tseng,et al.  On the convergence of the exponential multiplier method for convex programming , 1993, Math. Program..

[13]  Floyd J. Gould,et al.  A general saddle point result for constrained optimization , 1973, Math. Program..

[14]  Roman A. Polyak,et al.  Modified barrier functions (theory and methods) , 1992, Math. Program..

[15]  M. J. D. Powell,et al.  A method for nonlinear constraints in minimization problems , 1969 .

[16]  H. Mukai,et al.  A New Technique for Nonconvex Primal-Dual Decomposition of a Large-Scale Separable Optimization Prob , 1985 .

[17]  M. Fukushima,et al.  Nonlinear Proximal Decomposition Method for Convex Programming , 2000 .

[18]  R. D. Rupp.  A method for solving a quadratic optimal control problem , 1972 .

[19]  J. Betts.  A gradient projection-multiplier method for nonlinear programming , 1978 .

[20]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[21]  D. Bertsekas,et al.  Combined Primal–Dual and Penalty Methods for Convex Programming , 1976 .

[22]  John M. Mulvey,et al.  A New Scenario Decomposition Method for Large-Scale Stochastic Optimization , 1995, Oper. Res..

[23]  J. Betts.  An accelerated multiplier method for nonlinear programming , 1977 .

[24]  A. V. Levy,et al.  Use of the augmented penalty function in mathematical programming problems, part 1 , 1971 .

[25]  L. Grippo,et al.  A new augmented Lagrangian function for inequality constraints in nonlinear programming problems , 1982 .

[26]  Marc Teboulle,et al.  Convergence Rate Analysis of Nonquadratic Proximal Methods for Convex and Linear Programming , 1995, Math. Oper. Res..

[27]  Tao Pham Dinh,et al.  Proximal Decomposition on the Graph of a Maximal Monotone Operator , 1995, SIAM J. Optim..

[28]  Dimitri P. Bertsekas,et al.  Multiplier methods: A survey , 1975, at - Automatisierungstechnik.

[29]  H. Kunzi,et al.  Lectu re Notes in Economics and Mathematical Systems , 1975 .

[30]  E. Polak,et al.  A globally convergent, implementable multiplier method with automatic penalty limitation , 1980, 1980 19th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[31]  Marc Teboulle,et al.  Entropic Proximal Mappings with Applications to Nonlinear Programming , 1992, Math. Oper. Res..

[32]  Michael Zibulevsky,et al.  Penalty/Barrier Multiplier Methods for Convex Programming Problems , 1997, SIAM J. Optim..

[33]  A. V. Levy,et al.  On the method of multipliers for mathematical programming problems , 1972 .

[34]  S. Glad.  Properties of updating methods for the multipliers in augmented Lagrangians , 1979 .

[35]  J. Spingarn.  Partial inverse of a monotone operator , 1983 .

[36]  K. Narendra,et al.  Constrained optimization problems using multiplier methods , 1972 .

[37]  Elijah Polak,et al.  A quadratically convergent primal-dual algorithm with global convergence properties for solving optimization problems with equality constraints , 1975, Math. Program..

[38]  R. Tapia.  Diagonalized multiplier methods and quasi-Newton methods for constrained optimization , 1977 .

[39]  Piotr Tatjewski,et al.  Two-level primal-dual decomposition technique for large-scale nonconvex optimization problems with constraints , 1990 .

[40]  L. Grippo,et al.  A New Class of Augmented Lagrangians in Nonlinear Programming , 1979 .

[41]  R. Fletcher.  An Ideal Penalty Function for Constrained Optimization , 1975 .

[42]  Jonathan Eckstein.  Parallel alternating direction multiplier decomposition of convex programs , 1994 .

[43]  R. Byrd.  Local convergence of the diagonalized method of multipliers , 1978 .

[44]  R. D. Rupp.  On the combination of the multiplier method of Hestenes and Powell with Newton’s method , 1975 .

[45]  Igor Griva,et al.  Primal-Dual Nonlinear Rescaling Method for Convex Optimization , 2004 .

[46]  Paul Tseng,et al.  A Modified Forward-backward Splitting Method for Maximal Monotone Mappings 1 , 1998 .

[47]  P. Boggs,et al.  Augmented Lagrangians which are quadratic in the multiplier , 1980 .

[48]  R. Rockafellar.  The multiplier method of Hestenes and Powell applied to convex programming , 1973 .

[49]  Abdelouahed Hamdi,et al.  A re-scaled twin augmented Lagrangian algorithm for saddle point seeking , 2008 .

[50]  R. D. Rupp.  Approximation of the classical isoperimetric problem , 1972 .

[51]  Marc Teboulle,et al.  A Logarithmic-Quadratic Proximal Method for Variational Inequalities , 1999, Comput. Optim. Appl..

[52]  Wenyu Sun,et al.  New decomposition methods for solving variational inequality problems , 2003 .

[53]  A. V. Levy,et al.  Use of the augmented penalty function in mathematical programming problems, part 2 , 1971 .

[54]  Hai Yang,et al.  Decomposition Method for a Class of Monotone Variational Inequality Problems , 1999 .

[55]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[56]  R. D. Rupp.  A nonlinear optimal control minimization technique , 1973 .

[57]  Marc Teboulle,et al.  Interior Proximal and Multiplier Methods Based on Second Order Homogeneous Kernels , 1999, Math. Oper. Res..

[58]  Adamou Ouorou.  Decomposition proximale des problemes de multiflot a critere convexe. Application aux problemes de routage dans les reseaux de communication , 1995 .

[59]  Adam Ouorou,et al.  A Primal-Dual Algorithm for Monotropic Programming and its Application to Network Optimization , 2000, Comput. Optim. Appl..

[60]  O. Mangasarian.  Unconstrained Lagrangians in Nonlinear Programming , 1975 .

[61]  Jean-Pierre Dussault,et al.  Separable Augmented Lagrangian Algorithm with Multidimensional Scaling for Monotropic Programming , 2005 .

[62]  Paul Tseng,et al.  Alternating Projection-Proximal Methods for Convex Programming and Variational Inequalities , 1997, SIAM J. Optim..

[63]  D. Bertsekas.  On the method of multipliers for convex programming , 1975 .

[64]  Marc Teboulle,et al.  Entropic proximal decomposition methods for convex programs and variational inequalities , 2001, Math. Program..

[65]  Jonathan Eckstein,et al.  Nonlinear Proximal Point Algorithms Using Bregman Functions, with Applications to Convex Programming , 1993, Math. Oper. Res..

[66]  B. W. Kort.  RATE OF CONVERGENCE OF THE METHOD OF MULTIPLIERS WITH INEXACT MINIMIZATION , 1975 .

[67]  D. Bertsekas.  Approximation procedures based on the method of multipliers , 1977 .

[68]  R. Tyrrell Rockafellar,et al.  A dual approach to solving nonlinear programming problems by unconstrained optimization , 1973, Math. Program..

[69]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[70]  Roman A. Polyak,et al.  Log-Sigmoid Multipliers Method in Constrained Optimization , 2001, Ann. Oper. Res..

[71]  N. Watanabe,et al.  Decomposition in large system optimization using the method of multipliers , 1978 .

[72]  Jean-Pierre Dussault,et al.  Adaptive Scaling and Convergence Rates of a Separable Augmented Lagrangian Algorithm , 2000 .

[73]  Roman A. Polyak,et al.  Nonlinear Rescaling as Interior Quadratic Prox Method in Convex Optimization , 2006, Comput. Optim. Appl..

[74]  M C Ferris,et al.  Parallel Constraint Distribution , 1991, SIAM J. Optim..

[75]  Marc Teboulle,et al.  Convergence of Proximal-Like Algorithms , 1997, SIAM J. Optim..

[76]  Andrzej Ruszczynski,et al.  On Convergence of an Augmented Lagrangian Decomposition Method for Sparse Convex Optimization , 1995, Math. Oper. Res..

[77]  R. Glowinski,et al.  Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics , 1987 .

[78]  E. Polak.  Introduction to linear and nonlinear programming , 1973 .

[79]  S. Lucidi.  New results on a class of exact augmented Lagrangians , 1988 .

[80]  Piotr Tatjewski.  New dual-type decomposition algorithm for nonconvex separable optimization problems , 1989, Autom..

[81]  V. H. Nguyen,et al.  Family of perturbation methods for variational inequalities , 1996 .

[82]  R. Tyrrell Rockafellar,et al.  Convergence Rates in Forward-Backward Splitting , 1997, SIAM J. Optim..

[83]  P. C. Haarhoff,et al.  A New Method for the Optimization of a Nonlinear Function Subject to Nonlinear Constraints , 1970, Comput. J..

[84]  Marc Teboulle,et al.  A proximal-based decomposition method for convex minimization problems , 1994, Math. Program..

[85]  R. Rockafellar.  Augmented Lagrange Multiplier Functions and Duality in Nonconvex Programming , 1974 .

[86]  John M. Mulvey,et al.  A diagonal quadratic approximation method for large scale linear programs , 1992, Oper. Res. Lett..

[87]  K. Mårtensson,et al.  A new approach to constrained function optimization , 1973 .