A class of convergent primal-dual subgradient algorithms for decomposable convex programs

In this paper we develop a primal-dual subgradient algorithm for preferably decomposable, generally nondifferentiable, convex programming problems, under usual regularity conditions. The algorithm employs a Lagrangian dual function along with a suitable penalty function which satisfies a specified set of properties, in order to generate a sequence of primal and dual iterates for which some subsequence converges to a pair of primal-dual optimal solutions. Several classical types of penalty functions are shown to satisfy these specified properties. A geometric convergence rate is established for the algorithm under some additional assumptions. This approach has three principal advantages. Firstly, both primal and dual solutions are available which prove to be useful in several contexts. Secondly, the choice of step sizes, which plays an important role in subgradient optimization, is guided more determinably in this method via primal and dual information. Thirdly, typical subgradient algorithms suffer from the lack of an appropriate stopping criterion, and so the quality of the solution obtained after a finite number of steps is usually unknown. In contrast, by using the primal-dual gap, the proposed algorithm possesses a natural stopping criterion.

[1]  Philip Wolfe,et al.  Validation of subgradient optimization , 1974, Math. Program..

[2]  L. Khachiyan Polynomial algorithms in linear programming , 1980 .

[3]  G. M. Korpelevich The extragradient method for finding saddle points and other problems , 1976 .

[4]  R. Rockafellar The multiplier method of Hestenes and Powell applied to convex programming , 1973 .

[5]  I. J. Schoenberg,et al.  The Relaxation Method for Linear Inequalities , 1954, Canadian Journal of Mathematics.

[6]  D. Bertsekas COMBINED PRIMAL-DUAL AND PENALTY METHODS FOR CONSTRAINED MINIMIZATION* , 1975 .

[7]  R. Tyrrell Rockafellar,et al.  A dual approach to solving nonlinear programming problems by unconstrained optimization , 1973, Math. Program..

[8]  H. Sherali,et al.  On the choice of step size in subgradient optimization , 1981 .

[9]  C. Lemaréchal,et al.  ON A BUNDLE ALGORITHM FOR NONSMOOTH OPTIMIZATION , 1981 .

[10]  Arnoldo C. Hax,et al.  On the Solution of Convex Knapsack Problems with Bounded Variables. , 1977 .

[11]  Masao Fukushima,et al.  A descent algorithm for nonsmooth convex optimization , 1984, Math. Program..

[12]  A. M. Geoffrion Generalized Benders decomposition , 1972 .

[13]  Richard M. Karp,et al.  The Traveling-Salesman Problem and Minimum Spanning Trees , 1970, Oper. Res..

[14]  J. Goffin CONVERGENCE RESULTS IN A CLASS OF VARIABLE METRIC SUBGRADIENT METHODS , 1981 .

[15]  Philip E. Gill,et al.  Practical optimization , 1981 .

[16]  Richard M. Karp,et al.  The traveling-salesman problem and minimum spanning trees: Part II , 1971, Math. Program..

[17]  Krzysztof C. Kiwiel,et al.  An aggregate subgradient method for nonsmooth convex minimization , 1983, Math. Program..

[18]  R. Rockafellar Augmented Lagrange Multiplier Functions and Duality in Nonconvex Programming , 1974 .

[19]  Naum Z. Shor,et al.  Generalized Gradient Methods of Nondifferentiable Optimization Employing Space Dilatation Operations , 1982, ISMP.

[20]  Boris Polyak Minimization of unsmooth functionals , 1969 .

[21]  S. Agmon The Relaxation Method for Linear Inequalities , 1954, Canadian Journal of Mathematics.