Nesterov Step Reduced Gradient Algorithm for Convex Programming Problems

In this paper, we proposed an implementation of method of speed reduced gradient algorithm for optimizing a convex differentiable function subject to linear equality constraints and nonnegativity bounds on the variables. In particular, at each iteration, we compute a search direction by reduced gradient, and line search by bisection algorithm or Armijo rule. Under some assumption, the convergence rate of speed reduced gradient (SRG) algorithm is proven to be significantly better, both theoretically and practically. The algorithm of SRG are programmed by Matlab, and comparing by Frank-Wolfe algorithm some problems, the numerical results which show the efficient of our approach, we give also an application to ODE, optimal control, image and video co-localization and learning machine.

[1]  Fei-Fei Li,et al.  Efficient Image and Video Co-localization with Frank-Wolfe Algorithm , 2014, ECCV.

[2]  J. M. Martínez,et al.  Spectral Gradient Methods for Linearly Constrained Optimization , 2005 .

[3]  Panos M. Pardalos,et al.  A Collection of Test Problems for Constrained Global Optimization Algorithms , 1990, Lecture Notes in Computer Science.

[4]  Feng-Sheng Wang,et al.  Computation of optimal feedforward and feedback control by a modified reduced gradient method , 1999, Appl. Math. Comput..

[5]  A. N. Baushev,et al.  A Multidimensional Bisection Method for Minimizing Function over Simplex , 2007, World Congress on Engineering.

[6]  Abdelkrim El Mouatasim.  Implementation of reduced gradient with bisection algorithms for non-convex optimization problem via stochastic perturbation , 2017, Numerical Algorithms.

[7]  Ron S. Dembo,et al.  Dealing with degeneracy in reduced gradient algorithms , 1985, Math. Program..

[8]  Krzysztof C. Kiwiel,et al.  Proximity control in bundle methods for convex nondifferentiable minimization , 1990, Math. Program..

[9]  To Fu Ma,et al.  Reduced gradient method combined with augmented Lagrangian and barrier for the optimal power flow problem , 2008, Appl. Math. Comput..

[10]  Rachid Ellaia,et al.  A continuous approach to combinatorial optimization: application of water system pump operations , 2012, Optim. Lett..

[11]  Y. Nesterov.  A method for solving the convex programming problem with convergence rate O(1/k^2) , 1983 .

[12]  Claudio Sartori,et al.  A novel Frank-Wolfe algorithm. Analysis and applications to large-scale SVM training , 2013, Inf. Sci..

[13]  Klaus Schittkowski,et al.  More test examples for nonlinear programming codes , 1981 .

[14]  Rachid Ellaia,et al.  Stochastic perturbation of reduced gradient & GRG methods for nonconvex programming problems , 2014, Appl. Math. Comput..

[15]  Mark W. Schmidt,et al.  Block-Coordinate Frank-Wolfe Optimization for Structural SVMs , 2012, ICML.