Spectral projected subgradient with a momentum term for the Lagrangean dual approach

The Lagrangean dual problem, with a non-differentiable convex objective function, is usually solved by using the subgradient method, whose convergence is guaranteed if the optimal value of the dual objective function is known. In practice, this optimal value is approximated by a previously computed bound. In this work, we combine the subgradient method with a different choice of steplength, based on the recently developed spectral projected gradient method, that does not require either exact or approximated estimates of the optimal value. We also add a momentum term to the subgradient direction that accelerates the convergence process towards global solutions. To illustrate the behavior of our new algorithm we solve Lagrangean dual problems associated with integer programming problems. In particular, we present and discuss encouraging numerical results for set covering problems and generalized assignment problems.

[1]  Boris Polyak,et al.  Constrained minimization methods , 1966 .

[2]  L. Grippo,et al.  A nonmonotone line search technique for Newton's method , 1986 .

[3]  Naum Zuselevich Shor,et al.  Minimization Methods for Non-Differentiable Functions , 1985, Springer Series in Computational Mathematics.

[4]  Marcos Raydan,et al.  The Barzilai and Borwein Gradient Method for the Large Scale Unconstrained Minimization Problem , 1997, SIAM J. Optim..

[5]  Roger Fletcher,et al.  On the Barzilai-Borwein Method , 2005 .

[6]  Geoffrey E. Hinton,et al.  Experiments on Learning by Back Propagation. , 1986 .

[7]  John E. Beasley,et al.  A genetic algorithm for the generalised assignment problem , 1997, Comput. Oper. Res..

[8]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[9]  John N. Tsitsiklis,et al.  Introduction to linear optimization , 1997, Athena scientific optimization and computation series.

[10]  M. Raydan.  On the Barzilai and Borwein choice of steplength for the gradient method , 1993 .

[11]  G. Ross,et al.  Modeling Facility Location Problems as Generalized Assignment Problems , 1977 .

[12]  J. Beasley.  An algorithm for set covering problem , 1987 .

[13]  Paul Tseng,et al.  An Incremental Gradient(-Projection) Method with Momentum Term and Adaptive Stepsize Rule , 1998, SIAM J. Optim..

[14]  A. M. Geoffrion,et al.  Integer Programming Algorithms: A Framework and State-of-the-Art Survey , 1972 .

[15]  Jean-Louis Goffin,et al.  On convergence rates of subgradient optimization methods , 1977, Math. Program..

[16]  V. F. Dem'yanov,et al.  Nondifferentiable Optimization , 1985 .

[17]  Matteo Fischetti,et al.  A Heuristic Method for the Set Covering Problem , 1999, Oper. Res..

[18]  A. Goldstein.  Convex programming in Hilbert space , 1964 .

[19]  José Mario Martínez,et al.  Algorithm 813: SPG—Software for Convex-Constrained Optimization , 2001, TOMS.

[20]  Dimitri P. Bertsekas,et al.  Incremental Subgradient Methods for Nondifferentiable Optimization , 2001, SIAM J. Optim..

[21]  M. Solodov,et al.  Error Stability Properties of Generalized Gradient-Type Algorithms , 1998 .

[22]  John E. Beasley,et al.  OR-Library: Distributing Test Problems by Electronic Mail , 1990 .

[23]  José Mario Martínez,et al.  Spectral residual method without gradient information for solving large-scale nonlinear systems of equations , 2006, Math. Comput..

[24]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[25]  L. Liao,et al.  R-linear convergence of the Barzilai and Borwein gradient method , 2002 .

[26]  D. Bertsekas.  On the Goldstein-Levitin-Polyak gradient projection method , 1974, CDC 1974.

[27]  Alfredo N. Iusem,et al.  On the projected subgradient method for nonsmooth convex optimization in a Hilbert space , 1998, Math. Program..

[28]  José Mario Martínez,et al.  Nonmonotone Spectral Projected Gradient Methods on Convex Sets , 1999, SIAM J. Optim..

[29]  Philip Wolfe,et al.  Validation of subgradient optimization , 1974, Math. Program..