Stochastic Approximation for Consensus: A New Approach via Ergodic Backward Products

This paper considers both synchronous and asynchronous consensus algorithms with noisy measurements. For stochastic approximation based consensus algorithms, the existing convergence analysis with dynamic topologies heavily relies on quadratic Lyapunov functions, whose existence, however, may be difficult to ensure for switching directed graphs. Our main contribution is to introduce a new analytical approach. We first show a fundamental role of ergodic backward products for mean square consensus in algorithms with additive noise. Subsequently, we develop ergodicity results for backward products of degenerating stochastic matrices converging to a 0-1 matrix via a discrete time dynamical system approach. These results complement the existing ergodic theory of stochastic matrices and provide an effective tool for analyzing consensus problems. Under a joint connectivity assumption, our approach may deal with switching topologies, delayed measurements and correlated noises, and it does not require the double stochasticity condition typically assumed for the existence of quadratic Lyapunov functions.

[1]  P. Antsaklis,et al.  Asynchronous Consensus Protocols Using Nonlinear Paracontractions Theory , 2008, IEEE Transactions on Automatic Control.

[2]  Stephen P. Boyd,et al.  Randomized gossip algorithms , 2006, IEEE Transactions on Information Theory.

[3]  John Odentrantz,et al.  Markov Chains: Gibbs Fields, Monte Carlo Simulation, and Queues , 2000, Technometrics.

[4]  John N. Tsitsiklis,et al.  Asynchronous Stochastic Approximation and Q-Learning , 1994, Machine Learning.

[5]  Derivation of Consensus Algorithm Dynamics from Mean Field Stochastic Control NCE Equations , 2009 .

[6]  H. Kushner,et al.  Stochastic approximation algorithms for parallel and distributed processing , 1987 .

[7]  L. Elsner,et al.  On the convergence of asynchronous paracontractions with application to tomographic reconstruction from incomplete data , 1990 .

[8]  Richard M. Murray,et al.  Consensus problems in networks of agents with switching topology and time-delays , 2004, IEEE Transactions on Automatic Control.

[9]  L. Eisner.  Convergence of sequential and asynchronous nonlinear paracontractions , 2005 .

[10]  Asuman E. Ozdaglar,et al.  Convergence of rule-of-thumb learning rules in social networks , 2008, 2008 47th IEEE Conference on Decision and Control.

[11]  John N. Tsitsiklis,et al.  Weighted Gossip: Distributed Averaging using non-doubly stochastic matrices , 2010, 2010 IEEE International Symposium on Information Theory.

[12]  Martin J. Wainwright,et al.  Network-Based Consensus Averaging With General Noisy Channels , 2008, IEEE Transactions on Signal Processing.

[13]  E.M. Atkins,et al.  A survey of consensus problems in multi-agent coordination , 2005, Proceedings of the 2005, American Control Conference, 2005..

[14]  T. C. Aysal,et al.  Distributed Average Consensus With Dithered Quantization , 2008, IEEE Transactions on Signal Processing.

[15]  Jonathan H. Manton,et al.  Coordination and Consensus of Networked Agents with Noisy Measurements: Stochastic Algorithms and Asymptotic Behavior , 2009, SIAM J. Control. Optim..

[16]  Ruggero Carli,et al.  Communication constraints in the average consensus problem , 2008, Autom..

[17]  R. Carli,et al.  Average consensus on networks with transmission noise or quantization , 2007, 2007 European Control Conference (ECC).

[18]  J.N. Tsitsiklis,et al.  Convergence in Multiagent Coordination, Consensus, and Flocking , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[19]  Jorge Cortés,et al.  Distributed Strategies for Generating Weight-Balanced and Doubly Stochastic Digraphs , 2009, Eur. J. Control.

[20]  John N. Tsitsiklis,et al.  Distributed Asynchronous Deterministic and Stochastic Gradient Optimization Algorithms , 1984, 1984 American Control Conference.

[21]  J. Wolfowitz.  Products of indecomposable, aperiodic, stochastic matrices , 1963 .

[22]  Reza Olfati-Saber,et al.  Consensus and Cooperation in Networked Multi-Agent Systems , 2007, Proceedings of the IEEE.

[23]  V. Borkar.  Asynchronous Stochastic Approximations , 1998 .

[24]  Valerie Isham,et al.  Non‐Negative Matrices and Markov Chains , 1983 .

[25]  Soummya Kar,et al.  Distributed Consensus Algorithms in Sensor Networks With Imperfect Communication: Link Failures and Channel Noise , 2007, IEEE Transactions on Signal Processing.

[26]  R.W. Beard,et al.  Multi-agent Kalman consensus with relative uncertainty , 2005, Proceedings of the 2005, American Control Conference, 2005..

[27]  John S. Baras,et al.  Almost sure convergence to consensus in Markovian random graphs , 2008, 2008 47th IEEE Conference on Decision and Control.

[28]  Stephen P. Boyd,et al.  Distributed average consensus with least-mean-square deviation , 2007, J. Parallel Distributed Comput..

[29]  F. Cucker,et al.  Flocking in noisy environments , 2007, 0706.3343.

[30]  Tao Li,et al.  Consensus Conditions of Multi-Agent Systems With Time-Varying Topologies and Stochastic Communication Noises , 2010, IEEE Transactions on Automatic Control.

[31]  Vivek S. Borkar,et al.  Stochastic Approximation for Nonexpansive Maps: Application to Q-Learning Algorithms , 1997, SIAM J. Control. Optim..

[32]  Jonathan H. Manton,et al.  Stochastic Consensus Seeking With Noisy and Directed Inter-Agent Communication: Fixed and Randomly Varying Topologies , 2010, IEEE Transactions on Automatic Control.

[33]  Tao Li,et al.  Mean square average-consensus under measurement noises and fixed topologies: Necessary and sufficient conditions , 2009, Autom..

[34]  Alejandro Ribeiro,et al.  Consensus in Ad Hoc WSNs With Noisy Links—Part I: Distributed Estimation of Deterministic Signals , 2008, IEEE Transactions on Signal Processing.

[35]  Naomi Ehrich Leonard,et al.  Robustness of noisy consensus dynamics with directed communication , 2010, Proceedings of the 2010 American Control Conference.

[36]  Jie Lin,et al.  Coordination of groups of mobile autonomous agents using nearest neighbor rules , 2003, IEEE Trans. Autom. Control..

[37]  V. Anantharam,et al.  Structure theorems for partially asynchronous iterations of a nonnegative matrix with random delays , 1999 .

[38]  Behrouz Touri,et al.  Distributed consensus over network with noisy links , 2009, 2009 12th International Conference on Information Fusion.

[39]  Kenneth E. Barner,et al.  Convergence of Consensus Models With Stochastic Disturbances , 2010, IEEE Transactions on Information Theory.

[40]  R. Srikant,et al.  Quantized Consensus , 2006, 2006 IEEE International Symposium on Information Theory.

[41]  Randal W. Beard,et al.  Consensus seeking in multiagent systems under dynamically changing interaction topologies , 2005, IEEE Transactions on Automatic Control.

[42]  Gang George Yin,et al.  Consensus Formation in a Two-Time-Scale Markovian System , 2009, Multiscale Model. Simul..

[43]  John N. Tsitsiklis,et al.  On the Nonexistence of Quadratic Lyapunov Functions for Consensus Algorithms , 2007, IEEE Transactions on Automatic Control.

[44]  Subhrakanti Dey,et al.  Stochastic consensus over noisy networks with Markovian and arbitrary switches , 2010, Autom..

[45]  Srdjan S. Stankovic,et al.  Decentralized Parameter Estimation by Consensus Based Stochastic Approximation , 2007, IEEE Transactions on Automatic Control.

[46]  E. Seneta.  Non-negative Matrices and Markov Chains , 2008 .

[47]  Amr El Abbadi,et al.  Convergence Rates of Distributed Average Consensus With Stochastic Link Failures , 2010, IEEE Transactions on Automatic Control.