Monotonicity and restart in fast gradient methods

Fast gradient methods are known to be nonmonotone algorithms, and oscillations typically occur around the solution. To avoid this behavior, we propose in this paper a fast gradient method with restart, and analyze its convergence rate. The proposed algorithm bears similarities to other algorithms in the literature, but differs in a key point that enables theoretical convergence rate results. The efficiency of the proposed method is demonstrated by two numerical examples.

[1]  Michael Athans,et al.  Design of feedback control systems for stable plants with saturating actuators , 1988, Proceedings of the 27th IEEE Conference on Decision and Control.

[2]  Marc Teboulle,et al.  Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems , 2009, IEEE Transactions on Image Processing.

[3]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[4]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[5]  E. Mosca,et al.  Nonlinear control of constrained linear systems via predictive reference management , 1997, IEEE Trans. Autom. Control..

[6]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[7]  Stephen P. Boyd,et al.  Preconditioning in fast dual gradient methods , 2014, 53rd IEEE Conference on Decision and Control.

[8]  Yurii Nesterov,et al.  Smooth minimization of non-smooth functions , 2005, Math. Program..

[9]  P. Giselsson.  Improved Fast Dual Gradient Methods for Embedded Model Predictive Control , 2014 .

[10]  Stephen P. Boyd,et al.  A Splitting Method for Optimal Control , 2013, IEEE Transactions on Control Systems Technology.

[11]  Y. Nesterov.  A method for solving the convex programming problem with convergence rate O(1/k^2) , 1983 .

[12]  Emmanuel J. Candès,et al.  Adaptive Restart for Accelerated Gradient Schemes , 2012, Foundations of Computational Mathematics.

[13]  Alberto Bemporad,et al.  An Accelerated Dual Gradient-Projection Algorithm for Embedded Linear Model Predictive Control , 2014, IEEE Transactions on Automatic Control.