Gossip Algorithms for Distributed Signal Processing

Gossip algorithms are attractive for in-network processing in sensor networks because they do not require any specialized routing, there is no bottleneck or single point of failure, and they are robust to unreliable wireless network conditions. Recently, there has been a surge of activity in the computer science, control, signal processing, and information theory communities, developing faster and more robust gossip algorithms and deriving theoretical performance guarantees. This paper presents an overview of recent work in the area. We describe convergence rate results, which are related to the number of transmitted messages and thus the amount of energy consumed in the network for gossiping. We discuss issues related to gossiping over wireless links, including the effects of quantization and noise, and we illustrate the use of gossip algorithms for canonical signal processing tasks including distributed estimation, source localization, and compression.

[1]  George Cybenko,et al.  Dynamic Load Balancing for Distributed Memory Multiprocessors , 1989, J. Parallel Distributed Comput..

[2]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[3]  Martin Vetterli,et al.  Interval consensus: From quantized gossip to voting , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[4]  R. DeVore,et al.  Nonlinear approximation , 1998, Acta Numerica.

[5]  Gregory J. Pottie,et al.  Wireless integrated network sensors , 2000, Commun. ACM.

[6]  Alistair Sinclair,et al.  Improved Bounds for Mixing Rates of Markov Chains and Multicommodity Flow , 1992, Combinatorics, Probability and Computing.

[7]  D. Lucarelli,et al.  Model-based pose estimation by consensus , 2008, 2008 International Conference on Intelligent Sensors, Sensor Networks and Information Processing.

[8]  Angelia Nedic,et al.  Distributed and Recursive Parameter Estimation in Parametrized Linear State-Space Models , 2008, IEEE Transactions on Automatic Control.

[9]  Alejandro Ribeiro,et al.  Consensus in Ad Hoc WSNs With Noisy Links—Part I: Distributed Estimation of Deterministic Signals , 2008, IEEE Transactions on Signal Processing.

[10]  Yu Hen Hu,et al.  Energy based collaborative source localization using acoustic micro-sensor array , 2002, 2002 IEEE Workshop on Multimedia Signal Processing..

[11]  Richard M. Murray,et al.  Information flow and cooperative control of vehicle formations , 2004, IEEE Transactions on Automatic Control.

[12]  Radford M. Neal,et al.  ANALYSIS OF A NONREVERSIBLE MARKOV CHAIN SAMPLER , 2000 .

[13]  Alexandros G. Dimakis,et al.  Neighborhood gossip: Concurrent averaging through local interference , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[14]  Leonidas J. Guibas,et al.  Collaborative signal and information processing: an information-directed approach , 2003 .

[15]  Kevin Barraclough,et al.  I and i , 2001, BMJ : British Medical Journal.

[16]  Piyush Gupta,et al.  Critical Power for Asymptotic Connectivity in Wireless Networks , 1999 .

[17]  Jie Lin,et al.  Coordination of groups of mobile autonomous agents using nearest neighbor rules , 2003, IEEE Trans. Autom. Control..

[18]  Chee-Yee Chong,et al.  Sensor networks: evolution, opportunities, and challenges , 2003, Proc. IEEE.

[19]  Javad Lavaei,et al.  On quantized consensus by means of gossip algorithm - Part II: Convergence time , 2009, 2009 American Control Conference.

[20]  Mikhail Borisovich Nevelʹson,et al.  Stochastic Approximation and Recursive Estimation , 1976 .

[21]  Munther A. Dahleh,et al.  Distributed computation under bit constraints , 2008, 2008 47th IEEE Conference on Decision and Control.

[22]  Wei Ren,et al.  Information consensus in multivehicle cooperative control , 2007, IEEE Control Systems.

[23]  A. Dimakis,et al.  Geographic gossip: efficient aggregation for sensor networks , 2006, 2006 5th International Conference on Information Processing in Sensor Networks.

[24]  M.G. Rabbat,et al.  Generalized consensus computation in networked systems with erasure links , 2005, IEEE 6th Workshop on Signal Processing Advances in Wireless Communications, 2005..

[25]  W. Marsden.  I and J , 2012 .

[26]  José M. F. Moura,et al.  Weight Optimization for Consensus Algorithms With Correlated Switching Topology , 2009, IEEE Transactions on Signal Processing.

[27]  Soummya Kar,et al.  A mixed time-scale algorithm for distributed parameter estimation : Nonlinear observation models and imperfect communication , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[28]  Srdjan S. Stankovic,et al.  Decentralized Parameter Estimation by Consensus Based Stochastic Approximation , 2007, IEEE Transactions on Automatic Control.

[29]  José M. F. Moura,et al.  Distributing the Kalman Filter for Large-Scale Systems , 2007, IEEE Transactions on Signal Processing.

[30]  Yuval Rabani,et al.  Local divergence of Markov chains and the analysis of iterative load-balancing schemes , 1998, Proceedings 39th Annual Symposium on Foundations of Computer Science (Cat. No.98CB36280).

[31]  Deborah Estrin,et al.  Data-Centric Storage in Sensornets with GHT, a Geographic Hash Table , 2003, Mob. Networks Appl..

[32]  S. Muthukrishnan,et al.  Dynamic Load Balancing by Random Matchings , 1996, J. Comput. Syst. Sci..

[33]  M. Johansson,et al.  Faster Linear Iterations for Distributed Averaging , 2008 .

[34]  Gianluigi Ferrari.  Sensor networks : where theory meets practice , 2010 .

[35]  Richard M. Murray,et al.  Consensus problems in networks of agents with switching topology and time-delays , 2004, IEEE Transactions on Automatic Control.

[36]  Venkatesh Saligrama,et al.  Efficient In-Network Processing Through Local Ad-Hoc Information Coalescence , 2006, DCOSS.

[37]  Reza Olfati-Saber,et al.  Consensus and Cooperation in Networked Multi-Agent Systems , 2007, Proceedings of the IEEE.

[38]  Parameswaran Ramanathan,et al.  Distributed target classification and tracking in sensor networks , 2003 .

[39]  Matt Welsh,et al.  Simulating the power consumption of large-scale sensor network applications , 2004, SenSys '04.

[40]  R. Stephenson.  A and V , 1962, The British journal of ophthalmology.

[41]  Stephen P. Boyd,et al.  Fastest Mixing Markov Chain on a Graph , 2004, SIAM Rev..

[42]  U.A. Khan,et al.  A linear iterative algorithm for distributed sensor localization , 2008, 2008 42nd Asilomar Conference on Signals, Systems and Computers.

[43]  Asuman E. Ozdaglar,et al.  Constrained Consensus and Optimization in Multi-Agent Networks , 2008, IEEE Transactions on Automatic Control.

[45]  Isaac D. Scherson,et al.  An analysis of diffusive load-balancing , 1994, SPAA '94.

[46]  Bruno Sinopoli,et al.  Distributed control applications within sensor networks , 2003, Proc. IEEE.

[47]  Michael G. Rabbat,et al.  Optimization and Analysis of Distributed Averaging With Short Node Memory , 2009, IEEE Transactions on Signal Processing.

[48]  José M. F. Moura,et al.  Consensus Based Detection in Sensor Networks : Topology Optimization under Practical Constraints , 2007 .

[49]  Benjamin Van Roy,et al.  Consensus Propagation , 2005, IEEE Transactions on Information Theory.

[50]  T. C. Aysal,et al.  Distributed Average Consensus With Dithered Quantization , 2008, IEEE Transactions on Signal Processing.

[51]  Anand D. Sarwate,et al.  Broadcast Gossip Algorithms for Consensus , 2009, IEEE Transactions on Signal Processing.

[52]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[53]  Yu Hen Hu,et al.  Maximum likelihood multiple-source localization using acoustic energy measurements with wireless sensor networks , 2005, IEEE Trans. Signal Process..

[54]  Andreas Terzis,et al.  Distributed pose averaging in camera networks via consensus on SE(3) , 2008, 2008 Second ACM/IEEE International Conference on Distributed Smart Cameras.

[55]  A. Ortega,et al.  Energy-efficient data representation and routing for wireless sensor networks based on a distributed wavelet compression algorithm , 2006, 2006 5th International Conference on Information Processing in Sensor Networks.

[56]  Wen J. Li,et al.  Cluster-based distributed consensus , 2009, IEEE Trans. Wirel. Commun..

[57]  Aaron D. Wyner,et al.  The rate-distortion function for source coding with side information at the decoder , 1976, IEEE Trans. Inf. Theory.

[58]  J. Wolfowitz.  The rate distortion function for source coding with side information at the decoder , 1979 .

[59]  Jie Gao,et al.  Hierarchical Spatial Gossip for Multi-Resolution Representations in Sensor Networks , 2011, 2007 6th International Symposium on Information Processing in Sensor Networks.

[60]  Alfred O. Hero,et al.  Energy-based sensor network source localization via projection onto convex sets , 2005, IEEE Transactions on Signal Processing.

[61]  Ioannis D. Schizas,et al.  Stability analysis of the consensus-based distributed LMS algorithm , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[62]  Stephen P. Boyd,et al.  Fast linear iterations for distributed averaging , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[63]  Stephen P. Boyd,et al.  Randomized gossip algorithms , 2006, IEEE Transactions on Information Theory.

[64]  Viktor K. Prasanna,et al.  Energy-latency tradeoffs for data gathering in wireless sensor networks , 2004, IEEE INFOCOM 2004.

[65]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[66]  Ruggero Carli,et al.  Communication constraints in the average consensus problem , 2008, Autom..

[67]  Soummya Kar,et al.  Distributed Consensus Algorithms in Sensor Networks: Quantized Data and Random Link Failures , 2007, IEEE Transactions on Signal Processing.

[68]  D.J. Stilwell,et al.  Stochastic consensus over weighted directed networks , 2007, 2007 American Control Conference.

[69]  Bassam Bamieh,et al.  Distributed consensus with link failures as a structured stochastic uncertainty problem , 2008, 2008 46th Annual Allerton Conference on Communication, Control, and Computing.

[70]  Shlomo Shamai,et al.  Nested linear/Lattice codes for structured multiterminal binning , 2002, IEEE Trans. Inf. Theory.

[71]  G. Giannakis,et al.  Kalman Filtering in Wireless Sensor Networks , 2010, IEEE Control Systems.

[72]  Philippe Flajolet,et al.  Probabilistic Counting Algorithms for Data Base Applications , 1985, J. Comput. Syst. Sci..

[73]  Mehmet E. Yildiz,et al.  Coding With Side Information for Rate-Constrained Consensus , 2008, IEEE Transactions on Signal Processing.

[74]  Pascal Frossard,et al.  Polynomial Filtering for Fast Convergence in Distributed Consensus , 2008, IEEE Transactions on Signal Processing.

[75]  José M. F. Moura,et al.  A distributed sensor fusion algorithm for the inversion of sparse fields , 2009, 2009 Conference Record of the Forty-Third Asilomar Conference on Signals, Systems and Computers.

[76]  James A. Bucklew,et al.  Robust decentralized source localization via averaging , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[77]  M. Rabbat,et al.  Decentralized compression and predistribution via randomized gossiping , 2006, 2006 5th International Conference on Information Processing in Sensor Networks.

[78]  Rik Sarkar,et al.  Hierarchical spatial gossip for multi-resolution representations in sensor networks , 2007, IPSN '07.

[79]  Chai Wah Wu,et al.  Synchronization and convergence of linear dynamics in random directed networks , 2006, IEEE Transactions on Automatic Control.

[80]  Devavrat Shah,et al.  Information Dissemination via Gossip: Applications to Averaging and Coding , 2005 .

[81]  Yu Hen Hu,et al.  Energy-Based Collaborative Source Localization Using Acoustic Microsensor Array , 2003, EURASIP J. Adv. Signal Process..

[82]  Michael Rabbat,et al.  Distributed Average Consensus using Probabilistic Quantization , 2007, 2007 IEEE/SP 14th Workshop on Statistical Signal Processing.

[83]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[84]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[85]  Johannes Gehrke,et al.  Gossip-based computation of aggregate information , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[86]  John N. Tsitsiklis,et al.  Problems in decentralized decision making and computation , 1984 .

[87]  Alistair Sinclair,et al.  Improved Bounds for Mixing Rates of Marked Chains and Multicommodity Flow , 1992, LATIN.

[88]  Jack K. Wolf,et al.  Noiseless coding of correlated information sources , 1973, IEEE Trans. Inf. Theory.

[89]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[90]  Mung Chiang,et al.  The value of clustering in distributed estimation for sensor networks , 2005, 2005 International Conference on Wireless Networks, Communications and Mobile Computing.

[91]  John N. Tsitsiklis,et al.  Distributed asynchronous deterministic and stochastic gradient optimization algorithms , 1986 .

[92]  Antonio Ortega,et al.  Rate-distortion methods for image and video compression , 1998, IEEE Signal Process. Mag..

[93]  Richard M. Karp,et al.  Randomized rumor spreading , 2000, Proceedings 41st Annual Symposium on Foundations of Computer Science.

[94]  Alon Orlitsky,et al.  Average and randomized communication complexity , 1990, IEEE Trans. Inf. Theory.

[95]  Han-I Su,et al.  Distributed lossy averaging , 2010, IEEE Trans. Inf. Theory.

[96]  Stephen P. Boyd,et al.  Analysis and optimization of randomized gossip algorithms , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[97]  Kannan Ramchandran,et al.  Distributed compression in a dense microsensor network , 2002, IEEE Signal Process. Mag..

[98]  Robert Nowak,et al.  Distributed optimization in sensor networks , 2004, Third International Symposium on Information Processing in Sensor Networks, 2004. IPSN 2004.

[99]  John N. Tsitsiklis,et al.  Distributed Asynchronous Deterministic and Stochastic Gradient Optimization Algorithms , 1984, 1984 American Control Conference.

[100]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[101]  Alexandros G. Dimakis,et al.  Gossip along the way: order-optimal consensus through randomized path averaging , 2007 .

[102]  Mehran Mesbahi,et al.  Agreement over random networks , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[103]  Wen J. Li,et al.  Location-Aided Fast Distributed Consensus in Wireless Networks , 2010, IEEE Transactions on Information Theory.

[104]  Bruce M. Maggs,et al.  Approximate load balancing on dynamic and asynchronous networks , 1993, STOC.

[105]  A.S. Willsky,et al.  Distributed fusion in sensor networks , 2006, IEEE Signal Processing Magazine.

[106]  Ruggero Carli,et al.  Distributed Kalman filtering based on consensus strategies , 2008, IEEE Journal on Selected Areas in Communications.

[107]  R. Olfati-Saber,et al.  Distributed Kalman Filter with Embedded Consensus Filters , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[108]  Leonidas J. Guibas,et al.  Wireless sensor networks - an information processing approach , 2004, The Morgan Kaufmann series in networking.

[109]  Daniel A. Spielman,et al.  Accelerated Gossip Algorithms for Distributed Computation , 2006 .

[110]  Nicola Elia,et al.  Stabilization of linear systems with limited information , 2001, IEEE Trans. Autom. Control..

[111]  Alireza Tahbaz-Salehi,et al.  On consensus over random networks , 2006 .

[112]  V. Saligrama,et al.  Reliable Distributed Estimation with Intermittent Communications , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[113]  M. Alanyali,et al.  Distributed Detection in Sensor Networks With Packet Losses and Finite Capacity Links , 2006, IEEE Transactions on Signal Processing.

[114]  R. Srikant,et al.  Quantized Consensus , 2006, 2006 IEEE International Symposium on Information Theory.

[115]  Yu Hen Hu,et al.  Energy Based Acoustic Source Localization , 2003, IPSN.

[116]  Mehran Mesbahi,et al.  Distributed Linear Parameter Estimation in Sensor Networks based on Laplacian Dynamics Consensus Algorithm , 2006, 2006 3rd Annual IEEE Communications Society on Sensor and Ad Hoc Communications and Networks.

[117]  Sergio D. Servetto.  On the Feasibility of Large-Scale Wireless Sensor Networks , 2002 .

[118]  Michael Gastpar,et al.  Computation Over Multiple-Access Channels , 2007, IEEE Transactions on Information Theory.

[119]  R. Nowak,et al.  Compressed Sensing for Networked Data , 2008, IEEE Signal Processing Magazine.

[120]  Ruggero Carli,et al.  Distributed Kalman filtering using consensus strategies , 2007, 2007 46th IEEE Conference on Decision and Control.

[121]  Daniel Liberzon,et al.  On stabilization of linear systems with limited information , 2003, IEEE Trans. Autom. Control..

[122]  Angelia Nedic,et al.  Asynchronous gossip algorithms for stochastic optimization , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[123]  Wen J. Li,et al.  Location-Aided Fast Distributed Consensus , 2007, ArXiv.

[124]  Boris N. Oreshkin,et al.  Rates of convergence for greedy gossip with eavesdropping , 2008, 2008 46th Annual Allerton Conference on Communication, Control, and Computing.

[125]  Antonio Ortega,et al.  Energy-efficient data representation and routing for wireless sensor networks based on a distributed wavelet compression algorithm , 2006, 2006 5th International Conference on Information Processing in Sensor Networks.

[126]  Anna Scaglione,et al.  Scalable distributed Kalman filtering through consensus , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[127]  Soummya Kar,et al.  Distributed Sensor Localization in Random Environments Using Minimal Number of Anchor Nodes , 2008, IEEE Transactions on Signal Processing.

[128]  Soummya Kar,et al.  Topology for Distributed Inference on Graphs , 2006, IEEE Transactions on Signal Processing.

[129]  Alexandros G. Dimakis,et al.  The Impact of Mobility on Gossip Algorithms , 2012, IEEE Transactions on Information Theory.

[130]  J.H. Manton,et al.  Stochastic Lyapunov Analysis for Consensus Algorithms with Noisy Measurements , 2007, 2007 American Control Conference.

[131]  Soummya Kar,et al.  Distributed Consensus Algorithms in Sensor Networks With Imperfect Communication: Link Failures and Channel Noise , 2007, IEEE Transactions on Signal Processing.

[132]  U. Feige,et al.  Spectral Graph Theory , 2015 .

[133]  A. Scaglione,et al.  Differential Nested Lattice Encoding for Consensus Problems , 2007, 2007 6th International Symposium on Information Processing in Sensor Networks.

[134]  Sandro Zampieri,et al.  Randomized consensus algorithms over large scale networks , 2007 .

[135]  Han-I Su,et al.  Distributed Lossy Averaging , 2010, IEEE Transactions on Information Theory.

[136]  R.L. Moses,et al.  Locating the nodes: cooperative localization in wireless sensor networks , 2005, IEEE Signal Processing Magazine.

[137]  Fang Chen,et al.  Lifting Markov chains to speed up mixing , 1999, STOC '99.

[138]  Jinho Choi,et al.  Asymmetric power allocation to improve convergence rate of iterative receivers with soft cancellation , 2009, IEEE Communications Letters.

[139]  Devavrat Shah,et al.  Fast Distributed Algorithms for Computing Separable Functions , 2005, IEEE Transactions on Information Theory.

[140]  Ali H. Sayed,et al.  Diffusion Least-Mean Squares Over Adaptive Networks: Formulation and Performance Analysis , 2008, IEEE Transactions on Signal Processing.

[141]  Soummya Kar,et al.  Sensor Networks With Random Links: Topology Design for Distributed Consensus , 2007, IEEE Transactions on Signal Processing.