Success and failure of adaptation-diffusion algorithms for consensus in multi-agent networks

This paper investigates the problem of distributed stochastic approximation in multi-agent systems. The algorithm under study consists of two steps: a local stochastic approximation step and a gossip step which drives the network to a consensus. The gossip step uses row-stochastic matrices to weight network exchanges. We first prove the convergence of a distributed optimization algorithm, when the function to optimize may not be convex and the communication protocol is independent of the observations. In that case, we prove that the average estimate converges to a consensus; we also show that the set of limit points is not necessarily the set of the critical points of the function to optimize and is affected by the Perron eigenvector of the mean-matrix describing the communication protocol. Discussion about the success or failure of convergence to the minimizers of the function to optimize is also addressed. In a second part of the paper, we extend the convergence results to the more general context of distributed stochastic approximation.

[1]  Anand D. Sarwate,et al.  Broadcast Gossip Algorithms for Consensus , 2009, IEEE Transactions on Signal Processing.

[2]  Ali H. Sayed,et al.  Distributed processing over adaptive networks , 2007, 2007 9th International Symposium on Signal Processing and Its Applications.

[3]  Milos S. Stankovic,et al.  Decentralized Parameter Estimation by Consensus Based Stochastic Approximation , 2011, IEEE Trans. Autom. Control..

[4]  Angelia Nedic,et al.  Distributed Stochastic Subgradient Projection Algorithms for Convex Optimization , 2008, J. Optim. Theory Appl..

[5]  Jun Ye Yu,et al.  Networked optimization with adaptive communication , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[6]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[7]  G. Fort,et al.  Convergence of adaptive and interacting Markov chain Monte Carlo algorithms , 2011, 1203.3036.

[8]  Angelia Nedic,et al.  Asynchronous Broadcast-Based Convex Optimization Over a Network , 2011, IEEE Transactions on Automatic Control.

[9]  Eric Moulines,et al.  Stability of Stochastic Approximation under Verifiable Conditions , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[10]  Michael G. Rabbat,et al.  Push-Sum Distributed Dual Averaging for convex optimization , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[11]  Robert D. Nowak,et al.  Quantized incremental algorithms for distributed optimization , 2005, IEEE Journal on Selected Areas in Communications.

[12]  Angelia Nedic,et al.  Stochastic Gradient-Push for Strongly Convex Functions on Time-Varying Directed Graphs , 2014, IEEE Transactions on Automatic Control.

[13]  John N. Tsitsiklis,et al.  Distributed asynchronous deterministic and stochastic gradient optimization algorithms , 1986 .

[14]  Asuman E. Ozdaglar,et al.  Distributed Subgradient Methods for Multi-Agent Optimization , 2009, IEEE Transactions on Automatic Control.

[15]  John N. Tsitsiklis,et al.  Distributed Asynchronous Deterministic and Stochastic Gradient Optimization Algorithms , 1984, 1984 American Control Conference.

[16]  Ali H. Sayed,et al.  Incremental Adaptive Strategies Over Distributed Networks , 2007, IEEE Transactions on Signal Processing.

[17]  Pascal Bianchi,et al.  Asynchronous distributed optimization using a randomized alternating direction method of multipliers , 2013, 52nd IEEE Conference on Decision and Control.

[18]  Karl Henrik Johansson,et al.  Subgradient methods and consensus algorithms for solving convex optimization problems , 2008, 2008 47th IEEE Conference on Decision and Control.

[19]  R. M. Dudley,et al.  Real Analysis and Probability , 1989 .

[20]  Soummya Kar,et al.  Distributed Consensus Algorithms in Sensor Networks: Quantized Data and Random Link Failures , 2007, IEEE Transactions on Signal Processing.

[21]  M. Johansson,et al.  Distributed Optimization and Games: A Tutorial Overview , 2010 .

[22]  Stephen P. Boyd,et al.  Randomized gossip algorithms , 2006, IEEE Transactions on Information Theory.

[23]  Gersende Fort,et al.  Central Limit Theorems for Stochastic Approximation with controlled Markov chain dynamics , 2013, 1309.3116.

[24]  Angelia Nedic,et al.  Distributed optimization over time-varying directed graphs , 2013, 52nd IEEE Conference on Decision and Control.

[25]  Jie Chen,et al.  Multitask Diffusion Adaptation Over Networks , 2013, IEEE Transactions on Signal Processing.

[26]  Asuman E. Ozdaglar,et al.  Constrained Consensus and Optimization in Multi-Agent Networks , 2008, IEEE Transactions on Automatic Control.

[27]  Pascal Bianchi,et al.  A stochastic coordinate descent primal-dual algorithm and applications , 2014, 2014 IEEE International Workshop on Machine Learning for Signal Processing (MLSP).

[28]  Ali H. Sayed,et al.  Diffusion Adaptation Strategies for Distributed Optimization and Learning Over Networks , 2011, IEEE Transactions on Signal Processing.

[29]  Pascal Bianchi,et al.  Convergence of a Multi-Agent Projected Stochastic Gradient Algorithm for Non-Convex Optimization , 2011, IEEE Transactions on Automatic Control.

[30]  Johannes Gehrke,et al.  Gossip-based computation of aggregate information , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[31]  John N. Tsitsiklis,et al.  Problems in decentralized decision making and computation , 1984 .

[32]  P. Hall,et al.  Martingale Limit Theory and Its Application , 1980 .

[33]  Mérouane Debbah,et al.  Asynchronous alternating direction method of multipliers applied to the direct-current optimal power flow problem , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[34]  John N. Tsitsiklis,et al.  Weighted Gossip: Distributed Averaging using non-doubly stochastic matrices , 2010, 2010 IEEE International Symposium on Information Theory.

[35]  Angelia Nedic,et al.  Incremental Stochastic Subgradient Algorithms for Convex Optimization , 2008, SIAM J. Optim..

[36]  Pascal Bianchi,et al.  Performance of a Distributed Stochastic Approximation Algorithm , 2012, IEEE Transactions on Information Theory.

[37]  H. Kushner,et al.  Asymptotic properties of distributed and communication stochastic approximation algorithms , 1987 .

[38]  Pierre Priouret,et al.  Adaptive Algorithms and Stochastic Approximations , 1990, Applications of Mathematics.