Numerical Optimisation of Time-Varying Strongly Convex Functions Subject to Time-Varying Constraints

This paper analyses the performance of projected gradient descent on optimisation problems with cost functions and constraints that vary in discrete time. Specifically, strongly convex cost functions with Lipschitz gradient, and a sequence of convex constraints are assumed. Error bounds and suboptimality bounds are derived for a variety of cases, which show convergence to a steady-state. Conditions on the constraint sequence are also presented for guaranteeing finite-time feasibility, and for bounding the distance between successive minimisers. Numerical examples are then presented to validate the analytical results.

[1]  Alejandro Ribeiro,et al.  Network Integrity in Mobile Robotic Networks , 2013, IEEE Transactions on Automatic Control.

[2]  Qing Ling,et al.  Decentralized dynamic optimization through the alternating direction method of multipliers , 2013, SPAWC.

[3]  Slawomir Stanczak,et al.  A Distributed Subgradient Method for Dynamic Convex Optimization Problems Under Noisy Information Exchange , 2013, IEEE Journal of Selected Topics in Signal Processing.

[4]  Martin Zinkevich,et al.  Online Convex Programming and Generalized Infinitesimal Gradient Ascent , 2003, ICML.

[5]  Aryan Mokhtari,et al.  Optimization in Dynamic Environments : Improved Regret Rates for Strongly Convex Problems , 2016 .

[6]  D. Luenberger.  Optimization by Vector Space Methods , 1968 .

[7]  Dimitri P. Bertsekas,et al.  Convex Optimization Algorithms , 2015 .

[8]  Hao Yu,et al.  Online Convex Optimization with Time-Varying Constraints , 2017, 1702.04783.

[9]  Geert Leus,et al.  Distributed asynchronous time-varying constrained optimization , 2014, 2014 48th Asilomar Conference on Signals, Systems and Computers.

[10]  Ali H. Sayed,et al.  Mobile Adaptive Networks , 2011, IEEE Journal of Selected Topics in Signal Processing.

[11]  A. Yu. Popkov,et al.  Gradient Methods for Nonstationary Unconstrained Optimization Problems , 2005 .

[12]  Jonathan H. Manton,et al.  A framework for generalising the Newton method and other iterative methods from Euclidean space to manifolds , 2012, Numerische Mathematik.

[13]  Victor M. Zavala,et al.  Real-Time Nonlinear Optimization as a Generalized Equation , 2010, SIAM J. Control. Optim..

[14]  Andrea Simonetto.  Time-Varying Convex Optimization via Time-Varying Averaged Operators , 2017, 1704.07338.

[15]  Stephen J. Wright,et al.  Numerical Optimization (Springer Series in Operations Research and Financial Engineering) , 2000 .

[16]  R. Tyrrell Rockafellar,et al.  An Euler-Newton Continuation Method for Tracking Solution Trajectories of Parametric Variational Inequalities , 2013, SIAM J. Control. Optim..

[17]  Maojiao Ye,et al.  Distributed optimization for systems with time-varying quadratic objective functions , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[18]  James W. Daniel,et al.  Stability of the solution of definite quadratic programs , 1973, Math. Program..

[19]  Elad Hazan,et al.  Introduction to Online Convex Optimization , 2016, Found. Trends Optim..

[20]  Geert Leus,et al.  Double smoothing for time-varying distributed multiuser optimization , 2014, 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[21]  Usman A. Khan,et al.  Distributed dynamic optimization over directed graphs , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[22]  Maojiao Ye,et al.  Distributed Time-Varying Quadratic Optimization for Multiple Agents Under Undirected Graphs , 2017, IEEE Transactions on Automatic Control.

[23]  Aryan Mokhtari,et al.  A Class of Prediction-Correction Methods for Time-Varying Convex Optimization , 2015, IEEE Transactions on Signal Processing.

[24]  Alejandro Ribeiro,et al.  D-MAP: Distributed Maximum a Posteriori Probability Estimation of Dynamic Systems , 2013, IEEE Transactions on Signal Processing.