Improving Dynamic Regret in Distributed Online Mirror Descent Using Primal and Dual Information

We consider the problem of distributed online optimization, with a group of learners connected via a dynamic communication graph. The goal of the learners is to track the global minimizer of a sum of time-varying loss functions in a distributed manner. We propose a novel algorithm, termed Distributed Online Mirror Descent with Multiple Averaging Decision and Gradient Consensus (DOMDMADGC), which is based on mirror descent but incorporates multiple consensus averaging iterations over local gradients as well as local decisions. The key idea is to allow the local learners to collect a sufficient amount of global information, which enables them to more accurately approximation the time-varying global loss, so that they can closely track the dynamic global minimizer over time. We show that the dynamic regret of DOMD-MADGC is upper bounded by the path length, which is defined as the cumulative distance between successive minimizers. The resulting bound improves upon the bounds of existing distributed online algorithms and removes the explicit dependence on T .

[1]  Robert J. Ravier,et al.  Distributed Online Convex Optimization with Improved Dynamic Regret , 2019, 1911.05127.

[2]  Martin Zinkevich,et al.  Online Convex Programming and Generalized Infinitesimal Gradient Ascent , 2003, ICML.

[3]  Michael G. Rabbat,et al.  Multi-agent mirror descent for decentralized stochastic optimization , 2015, 2015 IEEE 6th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[4]  Yiguang Hong,et al.  Distributed Mirror Descent for Online Composite Optimization , 2020, IEEE Transactions on Automatic Control.

[5]  Qiong Wu,et al.  Distributed Mirror Descent over Directed Graphs , 2014, ArXiv.

[6]  Rebecca Willett,et al.  Online Convex Optimization in Dynamic Environments , 2015, IEEE Journal of Selected Topics in Signal Processing.

[7]  Omar Besbes,et al.  Non-Stationary Stochastic Optimization , 2013, Oper. Res..

[8]  Jinfeng Yi,et al.  Tracking Slowly Moving Clairvoyant: Optimal Dynamic Regret of Online Learning with True and Noisy Gradient , 2016, ICML.

[9]  Ketan Rajawat,et al.  Distributed Online Learning over Time-varying Graphs via Proximal Gradient Descent , 2019, 2019 IEEE 58th Conference on Decision and Control (CDC).

[10]  Francesco Orabona,et al.  Temporal Variability in Implicit Online Learning , 2020, NeurIPS.

[11]  Michael G. Rabbat,et al.  Efficient Distributed Online Prediction and Stochastic Optimization With Approximate Distributed Averaging , 2014, IEEE Transactions on Signal and Information Processing over Networks.

[12]  Xinlei Yi,et al.  Distributed Online Convex Optimization With an Aggregative Variable , 2020, IEEE Transactions on Control of Network Systems.

[13]  Shahin Shahrampour,et al.  Unconstrained Online Optimization: Dynamic Regret Analysis of Strongly Convex and Smooth Problems , 2020, ArXiv.

[14]  Elad Hazan,et al.  Introduction to Online Convex Optimization , 2016, Found. Trends Optim..

[15]  U. Feige,et al.  Spectral Graph Theory , 2015 .

[16]  Bahman Gharesifard,et al.  Distributed Online Convex Optimization on Time-Varying Directed Graphs , 2017, IEEE Transactions on Control of Network Systems.

[17]  Y. Singer,et al.  Logarithmic Regret Algorithms for Strongly Convex Repeated Games , 2007 .

[18]  Jorge Cortés,et al.  Distributed Online Convex Optimization Over Jointly Connected Digraphs , 2014, IEEE Transactions on Network Science and Engineering.

[19]  Francesco Orabona.  A Modern Introduction to Online Learning , 2019, ArXiv.

[20]  Angelia Nedic,et al.  Coordinate Dual Averaging for Decentralized Online Optimization With Nonseparable Global Objectives , 2015, IEEE Transactions on Control of Network Systems.

[21]  Shahin Shahrampour,et al.  Distributed Online Optimization in Dynamic Environments Using Mirror Descent , 2016, IEEE Transactions on Automatic Control.

[22]  Shahin Shahrampour,et al.  Online Optimization : Competing with Dynamic Comparators , 2015, AISTATS.

[23]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[24]  Gábor Lugosi,et al.  Prediction, learning, and games , 2006 .

[25]  Rong Jin,et al.  25th Annual Conference on Learning Theory Online Optimization with Gradual Variations , 2022 .

[26]  Mehran Mesbahi,et al.  Online distributed optimization via dual averaging , 2013, 52nd IEEE Conference on Decision and Control.

[27]  Shai Shalev-Shwartz,et al.  Online Learning and Online Convex Optimization , 2012, Found. Trends Mach. Learn..

[28]  Lin Xiao,et al.  Statistically Preconditioned Accelerated Gradient Method for Distributed Optimization , 2020, ICML.

[29]  Jinfeng Yi,et al.  Improved Dynamic Regret for Non-degenerate Functions , 2016, NIPS.

[30]  Ulrike von Luxburg,et al.  Hitting and commute times in large random neighborhood graphs , 2014, J. Mach. Learn. Res..

[31]  Rebecca Willett,et al.  Dynamical Models and tracking regret in online convex programming , 2013, ICML.

[32]  Aryan Mokhtari,et al.  Optimization in Dynamic Environments : Improved Regret Rates for Strongly Convex Problems , 2016 .

[33]  Lin Xiao,et al.  Dual Averaging Methods for Regularized Stochastic Learning and Online Optimization , 2009, J. Mach. Learn. Res..

[34]  Martin J. Wainwright,et al.  Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling , 2010, IEEE Transactions on Automatic Control.

[35]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[36]  Francesco Orabona,et al.  A closer look at temporal variability in dynamic online learning , 2021, ArXiv.

[37]  Long Wang,et al.  Online Distributed Optimization With Strongly Pseudoconvex-Sum Cost Functions , 2020, IEEE Transactions on Automatic Control.

[38]  Ambuj Tewari,et al.  Composite objective mirror descent , 2010, COLT 2010.

[39]  Elad Hazan,et al.  Logarithmic regret algorithms for online convex optimization , 2006, Machine Learning.

[40]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..