On the convergence of conditional epsilon-subgradient methods for convex programs and convex-concave saddle-point problems

Abstract The paper provides two contributions. First, we present new convergence results for conditional e -subgradient algorithms for general convex programs. The results obtained here extend the classical ones by Polyak [Sov. Math. Doklady 8 (1967) 593; USSR Comput. Math. Math. Phys. 9 (1969) 14; Introduction to Optimization, Optimization Software, New York, 1987] as well as the recent ones in [Math. Program. 62 (1993) 261; Eur. J. Oper. Res. 88 (1996) 382; Math. Program. 81 (1998) 23] to a broader framework. Secondly, we establish the application of this technique to solve non-strictly convex–concave saddle point problems, such as primal-dual formulations of linear programs. Contrary to several previous solution algorithms for such problems, a saddle-point is generated by a very simple scheme in which one component is constructed by means of a conditional e -subgradient algorithm, while the other is constructed by means of a weighted average of the (inexact) subproblem solutions generated within the subgradient method. The convergence result extends those of [Minimization Methods for Non-Differentiable Functions, Springer-Verlag, Berlin, 1985; Oper. Res. Lett. 19 (1996) 105; Math. Program. 86 (1999) 283] for Lagrangian saddle-point problems in linear and convex programming, and of [Int. J. Numer. Meth. Eng. 40 (1997) 1295] for a linear–quadratic saddle-point problem arising in topology optimization in contact mechanics.

[1]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[2]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[3]  M. Patriksson,et al.  Topology Optimization of Sheets in Contact by a Subgradient Method , 1997 .

[4]  M. Patriksson,et al.  Ergodic convergence in subgradient optimization , 1998 .

[5]  Michael Patriksson,et al.  Ergodic, primal convergence in dual subgradient schemes for convex programming , 1999, Mathematical programming.

[6]  Francisco Barahona,et al.  The volume algorithm: producing primal solutions with a subgradient method , 2000, Math. Program..

[7]  Alfredo N. Iusem,et al.  On the projected subgradient method for nonsmooth convex optimization in a Hilbert space , 1998, Math. Program..

[8]  Leon S. Lasdon,et al.  Optimization Theory of Large Systems , 1970 .

[9]  Claude Lemaréchal,et al.  Convergence of some algorithms for convex minimization , 1993, Math. Program..

[10]  R. Bellman,et al.  An Introduction to Minimax , 1976 .

[11]  Yu. M. Ermol’ev.  Methods of solution of nonlinear extremal problems , 1966 .

[12]  M. Patriksson,et al.  Conditional subgradient optimization -- Theory and applications , 1996 .

[13]  M. Solodov,et al.  Error Stability Properties of Generalized Gradient-Type Algorithms , 1998 .

[14]  V. F. Demʹi︠a︡nov,et al.  Introduction to minimax , 1976 .

[15]  Mokhtar S. Bazaraa,et al.  Nonlinear Programming: Theory and Algorithms , 1993 .

[16]  G. Nemhauser,et al.  Integer Programming , 2020 .

[17]  Adam Ouorou,et al.  A Primal-Dual Algorithm for Monotropic Programming and its Application to Network Optimization , 2000, Comput. Optim. Appl..

[18]  A. Propoi.  On the Theory of Max-Min , 1977 .

[19]  Donald W. Hearn,et al.  Lagrangian dual ascent by generalized linear programming , 1989 .

[20]  A. Ruszczynski,et al.  Perturbation Methods for Saddle Point Computation , 1994 .

[21]  V. F. Demʹi︠a︡nov,et al.  Introduction to minimax , 1976 .

[22]  R. Tyrrell Rockafellar,et al.  Primal-Dual Projected Gradient Algorithms for Extended Linear-Quadratic Programming , 1993, SIAM J. Optim..

[23]  Boris Polyak.  Minimization of unsmooth functionals , 1969 .

[24]  T. Larsson,et al.  A Lagrangean Relaxation Scheme for Structured Linear Programs With Application To Multicommodity Network Flows , 1997 .

[25]  丸山 徹.  Convex Analysisの二,三の進展について , 1977 .

[26]  Zhuang-Cheng Liu,et al.  A Dual Scheme for Traac Assignment Problems , 1997 .

[27]  O. Nelles,et al.  An Introduction to Optimization , 1996, IEEE Antennas and Propagation Magazine.

[28]  R. Rockafellar.  The theory of subgradients and its applications to problems of optimization : convex and nonconvex functions , 1981 .

[29]  J. A. Salvato.  John wiley & sons. , 1994, Environmental science & technology.

[30]  Markku Kallio,et al.  Large-Scale Convex Optimization Via Saddle Point Computation , 1999, Oper. Res..

[31]  Torbjörn Larsson,et al.  A dual scheme for traffic assignment problems , 1997 .

[32]  Hanif D. Sherali,et al.  Recovery of primal solutions when using subgradient optimization methods to solve Lagrangian duals of linear programs , 1996, Oper. Res. Lett..

[33]  Krzysztof C. Kiwiel,et al.  Proximal level bundle methods for convex nondifferentiable optimization, saddle-point problems and variational inequalities , 1995, Math. Program..

[34]  Naum Zuselevich Shor,et al.  Minimization Methods for Non-Differentiable Functions , 1985, Springer Series in Computational Mathematics.

[35]  X. Zhao,et al.  New Bundle Methods for Solving Lagrangian Relaxation Dual Problems , 2002 .