Relaxation Methods for Problems with Strictly Convex Costs and Linear Constraints

Consider the problem of minimizing a strictly convex possibly nondifferentiable and nonseparable cost subject to linear constraints. We propose a dual coordinate ascent method for this problem that uses inexact line search and either essentially cyclic or Gauss-Southwell order of coordinate relaxation. We show, under very weak conditions, that this method generates a sequence of primal vectors converging to the optimal primal solution. Under an additional regularity assumption, and assuming that the effective domain of the cost function is polyhedral, we show that a related sequence of dual vectors converges in cost to the optimal cost. If the constraint set has an interior point in the effective domain of the cost function, then this sequence of dual vectors is bounded and each of its limit points is an optimal dual solution. When the cost function is strongly convex, we show that the order of coordinate relaxation can become progressively more chaotic. These results significantly improve upon those obtained previously.

[1]  Clifford Hildreth,et al.  A quadratic programming procedure , 1957 .

[2]  D. D'Esopo,et al.  A convex programming procedure , 1959 .

[3]  L. Bregman.  The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming , 1967 .

[4]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[5]  C. Cryer.  The Solution of a Quadratic Programming Problem Using Systematic Overrelaxation , 1971 .

[6]  J. Darroch,et al.  Generalized Iterative Scaling for Log-Linear Models , 1972 .

[7]  E. Polak,et al.  Computational methods in optimization : a unified approach , 1972 .

[8]  M. J. D. Powell,et al.  On search directions for minimization algorithms , 1973, Math. Program..

[9]  R. Sargent,et al.  On the convergence of sequential minimization algorithms , 1973 .

[10]  T. Stern,et al.  A Class of Decentralized Routing Algorithms Using Relaxation , 1977, IEEE Trans. Commun..

[11]  Gabor T. Herman,et al.  A family of iterative quadratic optimization algorithms for pairs of inequalities, with application in diagnostic radiology , 1978 .

[12]  B. Korte,et al.  Minimum norm problems over transportation polytopes , 1980 .

[13]  N. F. Stewart,et al.  Bregman's balancing method , 1981 .

[14]  Y. Censor,et al.  An iterative row-action method for interval convex programming , 1981 .

[15]  R. Cottle,et al.  On the convergence of a block successive over-relaxation method for a class of linear complementarity problems , 1982 .

[16]  E. Jaynes.  On the rationale of maximum-entropy methods , 1982, Proceedings of the IEEE.

[17]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[18]  John G. Klincewicz,et al.  A Newton method for convex separable network flow problems , 1983, Networks.

[19]  Gene H. Golub,et al.  Matrix computations , 1983 .

[20]  Azuma Ohuchi,et al.  Lagrangian dual coordinatewise maximization algorithm for network transportation problems with quadratic costs , 1984, Networks.

[21]  Olvi L. Mangasarian.  Sparsity-preserving sor algorithms for separable quadratic and linear programming , 1984, Comput. Oper. Res..

[22]  J. Shore,et al.  Which is the better entropy expression for speech processing: -S log S or log S? , 1984 .

[23]  A. Barrett.  Network Flows and Monotropic Optimization. , 1984 .

[24]  S. Zenios,et al.  Relaxation techniques for strictly convex network problems , 1986 .

[25]  R. Cottle,et al.  A Lagrangean relaxation algorithm for the constrained matrix problem , 1986 .

[26]  D. Bertsekas,et al.  Distributed asynchronous relaxation methods for convex network flow problems , 1987 .

[27]  J. Pang,et al.  Iterative methods for large convex quadratic programs: a survey , 1987 .

[28]  D. Bertsekas,et al.  Relaxation methods for network flow problems with convex arc costs , 1987 .

[29]  Paul Tseng,et al.  Relaxation methods for problems with strictly convex separable costs and linear constraints , 1987, Math. Program..

[30]  Shih-Ping Han,et al.  A successive projection method , 1988, Math. Program..

[31]  Stavros A. Zenios,et al.  Nonlinear network optimization on a massively parallel connection machine , 1988 .

[32]  P. Tseng,et al.  On the convergence of a matrix splitting algorithm for the symmetric linear complementarity problem , 1989 .

[33]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[34]  John M. Mulvey,et al.  OR Practice - Large-Scale Nonlinear Network Models and Their Application , 1989, Oper. Res..

[35]  P. Tseng.  Dual ascent methods for problems with strictly convex costs and linear constraints: a unified approach , 1990 .

[36]  P. Tseng,et al.  On the convergence of a matrix splitting algorithm for the symmetric monotone linear complementarity problem , 1991 .