Eco-panda: A Computationally Economic, Geometrically Converging Dual Optimization Method on Time-varying Undirected Graphs

In this paper we consider distributed convex optimization over time-varying undirected graphs. We propose a linearized version of primarily averaged network dual ascent (PANDA) that keeps the advantages of PANDA while requiring less computational costs. The proposed method, economic primarily averaged network dual ascent (Eco-PANDA), provably converges at R-linear rate to the optimal point given that the agents’ objective functions are strongly convex and have Lipschitz continuous gradients. Therefore, the method is competitive, in terms of type of rate, with both DIGing and PANDA. The proposed method halves the communication costs of methods like DIGing while still converging R-linearly and having the same per iterate complexity.

[1]  Qing Ling,et al.  Decentralized learning for wireless communications and networking , 2015, ArXiv.

[2]  Ion Necoara,et al.  Parallel and distributed optimization methods for estimation and control in networks , 2011, 1302.3103.

[3]  Joakim Jald'en,et al.  A Geometrically Converging Dual Method for Distributed Optimization Over Time-Varying Graphs , 2018, IEEE Transactions on Automatic Control.

[4]  G.B. Giannakis,et al.  Distributed compression-estimation using wireless sensor networks , 2006, IEEE Signal Processing Magazine.

[5]  A. Yu. Popkov,et al.  Gradient Methods for Nonstationary Unconstrained Optimization Problems , 2005 .

[6]  Alejandro Ribeiro,et al.  Consensus in Ad Hoc WSNs With Noisy Links—Part I: Distributed Estimation of Deterministic Signals , 2008, IEEE Transactions on Signal Processing.

[7]  Angelia Nedić,et al.  Fast Convergence Rates for Distributed Non-Bayesian Learning , 2015, IEEE Transactions on Automatic Control.

[8]  Michael Rabbat,et al.  Decentralized source localization and tracking , 2004 .

[9]  Angelia Nedic,et al.  Optimal Algorithms for Distributed Optimization , 2017, ArXiv.

[10]  Qing Ling,et al.  EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimization , 2014, 1404.6264.

[11]  Wei Shi,et al.  Achieving Geometric Convergence for Distributed Optimization Over Time-Varying Graphs , 2016, SIAM J. Optim..

[12]  Sonia Martínez,et al.  Discrete-time dynamic average consensus , 2010, Autom..

[13]  Ziyang Meng,et al.  A Distributed Algorithm for Economic Dispatch Over Time-Varying Directed Networks With Delays , 2017, IEEE Transactions on Industrial Electronics.

[14]  Stephen P. Boyd,et al.  Distributed optimization for cooperative agents: application to formation flight , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[15]  Laurent Massoulié,et al.  Optimal Algorithms for Smooth and Strongly Convex Distributed Optimization in Networks , 2017, ICML.

[16]  Joakim Jaldén,et al.  PANDA: A Dual Linearly Converging Method for Distributed Optimization Over Time-Varying Undirected Graphs , 2018, 2018 IEEE Conference on Decision and Control (CDC).

[17]  Georgios B. Giannakis,et al.  Monitoring and Optimization for Power Grids: A Signal Processing Perspective , 2013, IEEE Signal Processing Magazine.

[18]  Ali Jadbabaie,et al.  Non-Bayesian Social Learning , 2011, Games Econ. Behav..