Online Learning With Inexact Proximal Online Gradient Descent Algorithms

We consider nondifferentiable dynamic optimization problems such as those arising in robotics and subspace tracking. Given the computational constraints and the time-varying nature of the problem, a low-complexity algorithm is desirable, while the accuracy of the solution may only increase slowly over time. We put forth the proximal online gradient descent (OGD) algorithm for tracking the optimum of a composite objective function comprising of a differentiable loss function and a nondifferentiable regularizer. An online learning framework is considered and the gradient of the loss function is allowed to be erroneous. Both, the gradient error as well as the dynamics of the function optimum or target are adversarial and the performance of the inexact proximal OGD is characterized in terms of its dynamic regret, expressed in terms of the cumulative error and path length of the target. The proposed inexact proximal OGD is generalized for application to large-scale problems where the loss function has a finite sum structure. In such cases, evaluation of the full gradient may not be viable and a variance reduced version is proposed that allows the component functions to be subsampled. The efficacy of the proposed algorithms is tested on the problem of formation control in robotics and on the dynamic foreground–background separation problem in video.

[1]  Jiming Chen,et al.  An Online Optimization Approach for Control and Communication Codesign in Networked Cyber-Physical Systems , 2013, IEEE Transactions on Industrial Informatics.

[2]  Lin Xiao,et al.  A Proximal Stochastic Gradient Method with Progressive Variance Reduction , 2014, SIAM J. Optim..

[3]  Ketan Rajawat,et al.  Time Varying optimization via Inexact Proximal Online Gradient Descent , 2018, 2018 52nd Asilomar Conference on Signals, Systems, and Computers.

[4]  Jinfeng Yi,et al.  Improved Dynamic Regret for Non-degenerate Functions , 2016, NIPS.

[5]  Angelia Nedic,et al.  On stochastic proximal-point method for convex-composite optimization , 2017, 2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[6]  Dimitri P. Bertsekas,et al.  Incremental Subgradient Methods for Nondifferentiable Optimization , 2001, SIAM J. Optim..

[7]  John N. Tsitsiklis,et al.  Gradient Convergence in Gradient methods with Errors , 1999, SIAM J. Optim..

[8]  Jingdong Wang,et al.  A Probabilistic Approach to Robust Matrix Factorization , 2012, ECCV.

[9]  Ketan Rajawat,et al.  Tracking Moving Agents via Inexact Online Gradient Descent Algorithm , 2017, IEEE Journal of Selected Topics in Signal Processing.

[10]  John C. S. Lui,et al.  Online Robust Subspace Tracking from Partial Information , 2011, ArXiv.

[11]  Jason C. Derenick,et al.  A convex optimization framework for multi-agent motion planning , 2009 .

[12]  Martin Kleinsteuber,et al.  pROST: a smoothed $$\ell _p$$ℓp-norm robust online subspace tracking method for background subtraction in video , 2013, Machine Vision and Applications.

[13]  Andrey Bernstein,et al.  Asynchronous and Distributed Tracking of Time-Varying Fixed Points , 2018, 2018 IEEE Conference on Decision and Control (CDC).

[14]  Namrata Vaswani,et al.  Recursive Robust PCA or Recursive Sparse Recovery in Large but Structured Noise , 2012, IEEE Transactions on Information Theory.

[15]  Martin Zinkevich,et al.  Online Convex Programming and Generalized Infinitesimal Gradient Ascent , 2003, ICML.

[16]  Tie-Yan Liu,et al.  Asynchronous Stochastic Proximal Optimization Algorithms with Variance Reduction , 2016, AAAI.

[17]  Shahin Shahrampour,et al.  Distributed Online Optimization in Dynamic Environments Using Mirror Descent , 2016, IEEE Transactions on Automatic Control.

[18]  Ketan Rajawat,et al.  Adaptive Low-Rank Matrix Completion , 2017, IEEE Transactions on Signal Processing.

[19]  Wolfgang Kellerer,et al.  Anomaly Detection and Identification in Large-scale Networks based on Online Time-structured Traffic Tensor Tracking , 2016 .

[20]  Mark W. Schmidt,et al.  Hybrid Deterministic-Stochastic Methods for Data Fitting , 2011, SIAM J. Sci. Comput..

[21]  John R. Spletzer,et al.  Convex Optimization Strategies for Coordinating Large-Scale Robot Formations , 2007, IEEE Transactions on Robotics.

[22]  John C. Duchi,et al.  Stochastic Methods for Composite Optimization Problems , 2017 .

[23]  Rebecca Willett,et al.  Online Convex Optimization in Dynamic Environments , 2015, IEEE Journal of Selected Topics in Signal Processing.

[24]  Angelia Nedic,et al.  Incremental Stochastic Subgradient Algorithms for Convex Optimization , 2008, SIAM J. Optim..

[25]  Namrata Vaswani,et al.  Online matrix completion and online robust PCA , 2015, 2015 IEEE International Symposium on Information Theory (ISIT).

[26]  Namrata Vaswani,et al.  Practical ReProCS for separating sparse and low-dimensional signal sequences from their sum — Part 2 , 2014, 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[27]  Dimitri P. Bertsekas,et al.  Incremental proximal methods for large scale convex optimization , 2011, Math. Program..

[28]  D. Bertsekas,et al.  Convergen e Rate of In remental Subgradient Algorithms , 2000 .

[29]  Aryan Mokhtari,et al.  Optimization in Dynamic Environments : Improved Regret Rates for Strongly Convex Problems , 2016 .

[30]  Feng Ruan,et al.  Stochastic Methods for Composite and Weakly Convex Optimization Problems , 2017, SIAM J. Optim..

[31]  Omar Besbes,et al.  Non-Stationary Stochastic Optimization , 2013, Oper. Res..

[32]  Shuicheng Yan,et al.  Online Robust PCA via Stochastic Optimization , 2013, NIPS.

[33]  Laura Balzano,et al.  Incremental gradient on the Grassmannian for online foreground and background separation in subsampled video , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Shahin Shahrampour,et al.  Online Optimization : Competing with Dynamic Comparators , 2015, AISTATS.

[35]  Geert Leus,et al.  On non-differentiable time-varying optimization , 2015, 2015 IEEE 6th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[36]  Francis Bach,et al.  SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives , 2014, NIPS.

[37]  M. Ani Hsieh,et al.  An Optimal Approach to Collaborative Target Tracking with Performance Guarantees , 2009, J. Intell. Robotic Syst..

[38]  Andrea Simonetto.  Time-Varying Convex Optimization via Time-Varying Averaged Operators , 2017, 1704.07338.

[39]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[40]  Jinfeng Yi,et al.  Tracking Slowly Moving Clairvoyant: Optimal Dynamic Regret of Online Learning with True and Noisy Gradient , 2016, ICML.

[41]  Thierry Bouwmans,et al.  Robust PCA via Principal Component Pursuit: A review for a comparative evaluation in video surveillance , 2014, Comput. Vis. Image Underst..