Distributed Optimization Over Markovian Switching Random Network

In this paper, we investigate the distributed convex optimization problem over a multi-agent system with Markovian switching communication networks. The objective function is the sum of each agent's local objective function, which cannot be known by other agents. The communication network is assumed to switch over a set of weight-balanced directed graphs with a Markovian property. We propose a consensus sub-gradient algorithm with two time-scale step-sizes to handle the uncertainty due to the Markovian switching topologies and the absence of global gradient information. With a proper selection of step-sizes, we prove the almost sure convergence of all agents' local estimates to the same optimal solution when the union graph of the Markovian network' states is strongly connected and the Markovian network is irreducible. Simulations are given for illustration of the results.

[1]  Han-Fu Chen,et al.  Asymptotic Properties of Primal-Dual Algorithm for Distributed Stochastic Optimization over Random Networks with Imperfect Communications , 2016, SIAM J. Control. Optim..

[2]  Ziyang Meng,et al.  A survey of distributed optimization , 2019, Annu. Rev. Control..

[3]  Han-Fu Chen.  Stochastic approximation and its applications , 2002 .

[4]  Feng Liu,et al.  Initialization-free distributed algorithms for optimal resource allocation with feasibility constraints and application to economic dispatch of power systems , 2015, Autom..

[5]  Anit Kumar Sahu,et al.  Convergence Rates for Distributed Stochastic Optimization Over Random Networks , 2018, 2018 IEEE Conference on Decision and Control (CDC).

[6]  Martin J. Wainwright,et al.  Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling , 2010, IEEE Transactions on Automatic Control.

[7]  N. Elia,et al.  Distributed multi-agent convex optimization over random digraphs , 2017, 2017 American Control Conference (ACC).

[8]  Angelia Nedic,et al.  Distributed Optimization for Control , 2018, Annu. Rev. Control. Robotics Auton. Syst..

[9]  Feng Liu,et al.  Distributed gradient algorithm for constrained optimization with application to load sharing in power systems , 2015, Syst. Control. Lett..

[10]  Yiguang Hong,et al.  Distributed resource allocation over random networks based on stochastic approximation , 2015, Syst. Control. Lett..

[11]  Angelia Nedic,et al.  Asynchronous Broadcast-Based Convex Optimization Over a Network , 2011, IEEE Transactions on Automatic Control.

[12]  Yiguang Hong,et al.  Quantized Subgradient Algorithm and Data-Rate Analysis for Distributed Optimization , 2014, IEEE Transactions on Control of Network Systems.

[13]  Qiang Zhang,et al.  Distributed Parameter Estimation Over Unreliable Networks With Markovian Switching Topologies , 2012, IEEE Transactions on Automatic Control.

[14]  Yiguang Hong,et al.  Distributed Continuous-Time Algorithm for Constrained Convex Optimizations via Nonsmooth Analysis Approach , 2015, IEEE Transactions on Automatic Control.

[15]  Yi Peng,et al.  Distributed cooperative optimization and its applications , 2016 .

[16]  Wei Shi,et al.  A Push-Pull Gradient Method for Distributed Optimization in Networks , 2018, 2018 IEEE Conference on Decision and Control (CDC).

[17]  Ali Sayed,et al.  Adaptation, Learning, and Optimization over Networks , 2014, Found. Trends Mach. Learn..

[18]  Giuseppe Notarstefano,et al.  Distributed Optimization for Smart Cyber-Physical Networks , 2019, Found. Trends Syst. Control..

[19]  Brian D. O. Anderson,et al.  Distributed Algorithms with Finite Data Rates that Solve Linear Equations , 2020, SIAM J. Optim..

[20]  Subhrakanti Dey,et al.  Stochastic consensus over noisy networks with Markovian and arbitrary switches , 2010, Autom..

[21]  Jiexiang Wang,et al.  Distributed Averaging With Random Network Graphs and Noises , 2017, IEEE Transactions on Information Theory.

[22]  Asuman E. Ozdaglar,et al.  Distributed Subgradient Methods for Multi-Agent Optimization , 2009, IEEE Transactions on Automatic Control.

[23]  Nicola Elia,et al.  Distributed Multiagent Convex Optimization Over Random Digraphs , 2020, IEEE Transactions on Automatic Control.

[24]  Choon Yik Tang,et al.  Gossip Algorithms for Convex Consensus Optimization Over Networks , 2010, IEEE Transactions on Automatic Control.

[25]  Asuman E. Ozdaglar,et al.  Distributed multi-agent optimization with state-dependent communication , 2010, Math. Program..

[26]  Georgios B. Giannakis,et al.  Distributed Optimal Power Flow for Smart Microgrids , 2012, IEEE Transactions on Smart Grid.

[27]  Qing Ling,et al.  EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimization , 2014, 1404.6264.

[28]  Lihua Xie,et al.  Convergence of Asynchronous Distributed Gradient Methods Over Stochastic Networks , 2018, IEEE Transactions on Automatic Control.

[29]  John S. Baras,et al.  Convergence Results for the Linear Consensus Problem under Markovian Random Graphs , 2013, SIAM J. Control. Optim..

[30]  Lihua Xie,et al.  Augmented distributed gradient methods for multi-agent optimization under uncoordinated constant stepsizes , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[31]  Angelia Nedic,et al.  Convergence Rate of Distributed Averaging Dynamics and Optimization in Networks , 2015, Found. Trends Syst. Control..

[32]  Lihua Xie,et al.  Kalman filtering over unreliable communication networks with bounded Markovian packet dropouts , 2009 .

[33]  Han-Fu Chen,et al.  Primal-dual algorithm for distributed constrained optimization , 2015, Syst. Control. Lett..

[34]  Asuman E. Ozdaglar,et al.  Distributed Subgradient Methods for Convex Optimization Over Random Networks , 2011, IEEE Transactions on Automatic Control.