Distributed Stochastic Approximation: Weak Convergence and Network Design

This paper studies distributed stochastic approximation algorithms based on broadcast gossip on communication networks represented by digraphs. Weak convergence of these algorithms is proved, and an associated ordinary differential equation (ODE) is formulated connecting convergence points with local objective functions and network properties. Using these results, a methodology is proposed for network design, aimed at achieving the desired asymptotic behavior at consensus. Convergence rate of the algorithm is also analyzed and further improved using an attached stochastic differential equation. Simulation results illustrate the theoretical concepts.

[1]  Stephen P. Boyd,et al.  Randomized gossip algorithms , 2006, IEEE Transactions on Information Theory.

[2]  Ali H. Sayed,et al.  Diffusion Strategies Outperform Consensus Strategies for Distributed Estimation Over Adaptive Networks , 2012, IEEE Transactions on Signal Processing.

[3]  Behrouz Touri,et al.  Product of Random Stochastic Matrices , 2011, IEEE Transactions on Automatic Control.

[4]  Pierre Priouret,et al.  Adaptive Algorithms and Stochastic Approximations , 1990, Applications of Mathematics.

[5]  Robert J. Plemmons,et al.  Nonnegative Matrices in the Mathematical Sciences , 1979, Classics in Applied Mathematics.

[6]  Anand D. Sarwate,et al.  Broadcast Gossip Algorithms for Consensus , 2009, IEEE Transactions on Signal Processing.

[7]  Han-Fu Chen.  Stochastic approximation and its applications , 2002 .

[8]  H. Kushner,et al.  Stochastic approximation algorithms for parallel and distributed processing , 1987 .

[9]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[10]  Minyi Huang,et al.  Stochastic Approximation for Consensus: A New Approach via Ergodic Backward Products , 2012, IEEE Transactions on Automatic Control.

[11]  Pascal Bianchi,et al.  On the Convergence of a Multi-Agent Projected Stochastic Gradient Algorithm , 2011, ArXiv.

[12]  John N. Tsitsiklis,et al.  Distributed Asynchronous Deterministic and Stochastic Gradient Optimization Algorithms , 1984, 1984 American Control Conference.

[13]  Angelia Nedic,et al.  Distributed optimization over time-varying directed graphs , 2013, 52nd IEEE Conference on Decision and Control.

[14]  Milos S. Stankovic,et al.  Consensus based overlapping decentralized estimation with missing observations and communication faults , 2009, Autom..

[15]  Angelia Nedic,et al.  Asynchronous Broadcast-Based Convex Optimization Over a Network , 2011, IEEE Transactions on Automatic Control.

[16]  John N. Tsitsiklis,et al.  Problems in decentralized decision making and computation , 1984 .

[17]  John N. Tsitsiklis,et al.  Convergence of Type-Symmetric and Cut-Balanced Consensus Seeking Systems , 2011, IEEE Transactions on Automatic Control.

[18]  Christoforos N. Hadjicostis,et al.  Distributed Matrix Scaling and Application to Average Consensus in Directed Graphs , 2013, IEEE Transactions on Automatic Control.

[19]  H. Kushner,et al.  Stochastic Approximation and Recursive Algorithms and Applications , 2003 .

[20]  V. Borkar.  Asynchronous Stochastic Approximations , 1998 .

[21]  Asuman E. Ozdaglar,et al.  Distributed Subgradient Methods for Multi-Agent Optimization , 2009, IEEE Transactions on Automatic Control.

[22]  Asuman E. Ozdaglar,et al.  Distributed Subgradient Methods for Convex Optimization Over Random Networks , 2011, IEEE Transactions on Automatic Control.

[23]  Pascal Bianchi,et al.  Performance of a Distributed Stochastic Approximation Algorithm , 2012, IEEE Transactions on Information Theory.

[24]  Behrouz Touri,et al.  On backward product of stochastic matrices , 2011, Autom..

[25]  J.N. Tsitsiklis,et al.  Convergence in Multiagent Coordination, Consensus, and Flocking , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[26]  Pascal Bianchi,et al.  Convergence of a Multi-Agent Projected Stochastic Gradient Algorithm for Non-Convex Optimization , 2011, IEEE Transactions on Automatic Control.

[27]  Srdjan S. Stankovic,et al.  Decentralized Parameter Estimation by Consensus Based Stochastic Approximation , 2007, IEEE Transactions on Automatic Control.

[28]  Angelia Nedic,et al.  Distributed Stochastic Subgradient Projection Algorithms for Convex Optimization , 2008, J. Optim. Theory Appl..

[29]  H. Kushner,et al.  Asymptotic properties of distributed and communication stochastic approximation algorithms , 1987 .

[30]  Asuman E. Ozdaglar,et al.  Constrained Consensus and Optimization in Multi-Agent Networks , 2008, IEEE Transactions on Automatic Control.