Analysis of accelerated gossip algorithms

This paper investigates accelerated gossip algorithms for distributed computations in networks where shift-registers are utilized at each node. By using tools from matrix analysis, we prove the existence of the desired acceleration and establish the fastest rate of convergence in expectation for two-register symmetric gossip. Some classes of networks with regular graph topologies are studied in detail to validate the analytical results by comparison with existing empirical data. We also analyze convergence of second moment and provide a necessary condition for convergence in multi-register symmetric gossip. The proposed approach has the potential to be applied to the more challenging open problem of asymmetric gossip.

[1]  Daniel A. Spielman,et al.  Accelerated Gossip Algorithms for Distributed Computation , 2006 .

[2]  Shaoshuai Mou,et al.  Deterministic gossiping with a periodic protocol , 2010, 49th IEEE Conference on Decision and Control (CDC).

[3]  M. Johansson,et al.  Faster Linear Iterations for Distributed Averaging , 2008 .

[4]  Fenghua He,et al.  Periodic Gossiping , 2011 .

[5]  D. Cvetkovic,et al.  Spectra of graphs : theory and application , 1995 .

[6]  B. Anderson,et al.  The Contraction Coefficient of a Complete Gossip Sequence , 2010 .

[7]  R. Varga,et al.  Chebyshev semi-iterative methods, successive overrelaxation iterative methods, and second order Richardson iterative methods , 1961 .

[8]  Stephen P. Boyd,et al.  Randomized gossip algorithms , 2006, IEEE Transactions on Information Theory.

[9]  J. Gillis,et al.  Matrix Iterative Analysis , 1961 .

[10]  S. Muthukrishnan,et al.  First- and Second-Order Diffusive Methods for Rapid, Coarse, Distributed Load Balancing , 1996, Theory of Computing Systems.

[11]  Noga Alon,et al.  Eigenvalues and expanders , 1986, Comb..

[12]  J.N. Tsitsiklis,et al.  Convergence Rates in Distributed Consensus and Averaging , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[13]  Benjamin Van Roy,et al.  Consensus Propagation , 2005, IEEE Transactions on Information Theory.

[14]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[15]  Stephen P. Boyd,et al.  Fast linear iterations for distributed averaging , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[16]  N. Biggs.  Algebraic Graph Theory , 1974 .

[17]  Joel Friedman,et al.  A proof of Alon's second eigenvalue conjecture and related problems , 2004, ArXiv.

[18]  John N. Tsitsiklis,et al.  Convergence Speed in Distributed Consensus and Averaging , 2009, SIAM J. Control. Optim..

[19]  Mathew D. Penrose,et al.  Random Geometric Graphs , 2003 .

[20]  Alexandros G. Dimakis,et al.  Order-Optimal Consensus Through Randomized Path Averaging , 2010, IEEE Transactions on Information Theory.

[21]  Stephen P. Boyd,et al.  Gossip algorithms: design, analysis and applications , 2005, Proceedings IEEE 24th Annual Joint Conference of the IEEE Computer and Communications Societies..

[22]  Brian D. O. Anderson,et al.  Reaching a Consensus in a Dynamically Changing Environment: Convergence Rates, Measurement Delays, and Asynchronous Events , 2008, SIAM J. Control. Optim..

[23]  John N. Tsitsiklis,et al.  On Distributed Averaging Algorithms and Quantization Effects , 2008, IEEE Trans. Autom. Control..

[24]  D. Cvetkovic,et al.  Spectra of Graphs: Theory and Applications , 1997 .

[25]  Devavrat Shah,et al.  Computing separable functions via gossip , 2005, PODC '06.

[26]  A. Dimakis,et al.  Geographic gossip: efficient aggregation for sensor networks , 2006, 2006 5th International Conference on Information Processing in Sensor Networks.

[27]  Shaoshuai Mou,et al.  Deterministic Gossiping , 2011, Proceedings of the IEEE.

[28]  Michael G. Rabbat,et al.  Optimization and Analysis of Distributed Averaging With Short Node Memory , 2009, IEEE Transactions on Signal Processing.

[29]  R.M. Murray,et al.  Asynchronous Distributed Averaging on Communication Networks , 2007, IEEE/ACM Transactions on Networking.

[30]  T. C. Aysal,et al.  Accelerated Distributed Average Consensus via Localized Node State Prediction , 2009, IEEE Transactions on Signal Processing.

[31]  Brian D. O. Anderson,et al.  Analysis of accelerated gossip algorithms , 2009, CDC.

[32]  M. Cao,et al.  A Lower Bound on Convergence of a Distributed Network Consensus Algorithm , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[33]  Brian D. O. Anderson,et al.  Convergence of Periodic Gossiping Algorithms , 2010 .

[34]  Soummya Kar,et al.  Gossip Algorithms for Distributed Signal Processing , 2010, Proceedings of the IEEE.

[35]  Johannes Gehrke,et al.  Gossip-based computation of aggregate information , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[36]  H GolubGene,et al.  Chebyshev semi-iterative methods, successive overrelaxation iterative methods, and second order Richardson iterative methods , 1961 .