On non-differentiable time-varying optimization

We consider non-differentiable convex optimization problems that vary continuously in time and we propose algorithms that sample these problems at specific time instances and generate a sequence of converging near-optimal decision variables. This sequence converges up to a bounded error to the solution trajectory of the time-varying non-differentiable problems. We illustrate through analytical examples and a realistic numerical simulation the benefit of the algorithms in signal processing applications, e.g., for reconstructing time-varying sparse signals.

[1]  Aryan Mokhtari,et al.  A Class of Prediction-Correction Methods for Time-Varying Convex Optimization , 2015, IEEE Transactions on Signal Processing.

[2]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[3]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[4]  Namrata Vaswani,et al.  Recursive Recovery of Sparse Signal Sequences From Compressive Measurements: A Review , 2016, IEEE Transactions on Signal Processing.

[5]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[6]  A. Yu. Popkov,et al.  Gradient Methods for Nonstationary Unconstrained Optimization Problems , 2005 .

[7]  R. Tyrrell Rockafellar,et al.  An Euler-Newton Continuation Method for Tracking Solution Trajectories of Parametric Variational Inequalities , 2013, SIAM J. Control. Optim..

[8]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[9]  Stephen P. Boyd,et al.  A Primer on Monotone Operator Methods , 2015 .

[10]  Qing Ling,et al.  Decentralized Dynamic Optimization Through the Alternating Direction Method of Multipliers , 2013, IEEE Transactions on Signal Processing.

[11]  D. Bertsekas,et al.  Convergen e Rate of In remental Subgradient Algorithms , 2000 .

[12]  Marius Pesavento,et al.  An Online Parallel and Distributed Algorithm for Recursive Estimation of Sparse Signals , 2015 .

[13]  Georgios B. Giannakis,et al.  Online Adaptive Estimation of Sparse Signals: Where RLS Meets the $\ell_1$ -Norm , 2010, IEEE Transactions on Signal Processing.

[14]  Shai Shalev-Shwartz,et al.  Online Learning and Online Convex Optimization , 2012, Found. Trends Mach. Learn..

[15]  Euhanna Ghadimi,et al.  Accelerating Convergence of Large-scale Optimization Algorithms , 2015 .

[16]  Thomas B. Schön,et al.  2015 IEEE 6th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, CAMSAP 2015 , 2016 .

[17]  Wotao Yin,et al.  On the Global and Linear Convergence of the Generalized Alternating Direction Method of Multipliers , 2016, J. Sci. Comput..

[18]  Arindam Banerjee,et al.  Online Alternating Direction Method , 2012, ICML.

[19]  Alexander G. Gray,et al.  Stochastic ADMM for Nonsmooth Optimization , 2012, ArXiv.

[20]  S. Uryasev,et al.  Stochastic optimization : Algorithms and Applications , 2001 .

[21]  s-taiji.  Dual Averaging and Proximal Gradient Descent for Online Alternating Direction Multiplier Method , 2013 .