On the Computational Power of Simple Dynamics

This work presents a set of analytical results regarding some elementary randomized protocols, called \emph{dynamics}, for solving some fundamental computational problems. New techniques for analyzing the processes that arise from such dynamics are presented, together with concrete examples on how to build efficient and robust distributed algorithms for some important tasks using these processes as a black-box. More specifically, we analyze several dynamics such as the 3-Majority, the Averaging and the Undecided-State ones, and we show how to use them to solve fundamental problems such as plurality consensus, community detection (including the reconstruction problem in the stochastic block model), and bit dissemination (rumor spreading). We focus mainly on unstructured and random interaction models, and we also deal with scenarios in which the communication is affected by noise or when a self-stabilizing protocol is required.

[1]  Dariusz R. Kowalski,et al.  Distributed agreement with optimal communication complexity , 2010, SODA '10.

[2]  Amin Coja-Oghlan,et al.  Graph Partitioning via Adaptive Spectral Techniques , 2009, Combinatorics, Probability and Computing.

[3]  Upendra Dave,et al.  Applied Probability and Queues , 1987 .

[4]  Christian Scheideler,et al.  Stabilizing consensus with the power of two choices , 2011, SPAA '11.

[5]  Jehoshua Bruck,et al.  Programmability of Chemical Reaction Networks , 2009, Algorithmic Bioprocesses.

[6]  R. Baxter Exactly solved models in statistical mechanics , 1982 .

[7]  Frank McSherry,et al.  Spectral partitioning of random graphs , 2001, Proceedings 2001 IEEE International Conference on Cluster Computing.

[8]  Stefan Grosskinsky Warwick,et al.  Interacting Particle Systems , 2016 .

[9]  Allan Borodin,et al.  Adversarial queuing theory , 2001, JACM.

[10]  Joel Friedman,et al.  The Relativized Second Eigenvalue Conjecture of Alon , 2014, ArXiv.

[11]  Yves Métivier,et al.  An optimal bit complexity randomized distributed MIS algorithm , 2011, Distributed Computing.

[12]  Shlomi Dolev,et al.  Direction election in flocking swarms , 2010, DIALM-POMC '10.

[13]  Berthold Vöcking,et al.  How asymmetry helps load balancing , 1999, JACM.

[14]  Edsger W. Dijkstra,et al.  Self-stabilizing systems in spite of distributed control , 1974, CACM.

[15]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[16]  Michael Mitzenmacher,et al.  The Power of Two Choices in Randomized Load Balancing , 2001, IEEE Trans. Parallel Distributed Syst..

[17]  Richard M. Karp,et al.  Randomized rumor spreading , 2000, Proceedings 41st Annual Symposium on Foundations of Computer Science.

[18]  A. Frieze Spectral Techniques, Semidefinite Programs, and Random Graphs , 2005 .

[19]  Luca Trevisan,et al.  Stabilizing Consensus with Many Opinions , 2015, SODA.

[20]  Moez Draief,et al.  Distributed Multivalued Consensus , 2012, ISCIS.

[21]  David Eisenstat,et al.  A simple population protocol for fast robust approximate majority , 2007, Distributed Computing.

[22]  Michael Kearns,et al.  Biased Voting and the Democratic Primary Problem , 2008, WINE.

[23]  V. Climenhaga Markov chains and mixing times , 2013 .

[24]  Michael J. Fischer,et al.  Computation in networks of passively mobile finite-state sensors , 2004, PODC '04.

[25]  Pietro Liò,et al.  Towards real-time community detection in large networks. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[26]  Luca Cardelli,et al.  The Cell Cycle Switch Computes Approximate Majority , 2012, Scientific Reports.

[27]  Sariel Har-Peled,et al.  Random Walks , 2021, Encyclopedia of Social Network Analysis and Mining.

[28]  Shlomi Dolev,et al.  Self Stabilization , 2004, J. Aerosp. Comput. Inf. Commun..

[29]  Yongxiang Ruan,et al.  Binary consensus with soft information processing in cooperative networks , 2008, 2008 47th IEEE Conference on Decision and Control.

[30]  Andrea E. F. Clementi,et al.  Rumor spreading in random evolving graphs , 2013, Random Struct. Algorithms.

[31]  Luca Cardelli,et al.  Chemical reaction network designs for asynchronous logic circuits , 2016, Natural Computing.

[32]  Chen Avin,et al.  How to Explore a Fast-Changing World (Cover Time of a Simple Random Walk on Evolving Graphs) , 2008, ICALP.

[33]  Emmanuel Abbe,et al.  Detection in the stochastic block model with multiple clusters: proof of the achievability conjectures, acyclic BP, and the information-computation gap , 2015, ArXiv.

[34]  Kyomin Jung,et al.  Distributed ranking in networks with limited memory and communication , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[35]  Hagit Attiya,et al.  Optimal clock synchronization under different delay assumptions , 1993, PODC '93.

[36]  Noah E. Friedkin,et al.  Social influence and opinions , 1990 .

[37]  Christoph Lenzen,et al.  Synchronous counting and computational algorithm design , 2013, J. Comput. Syst. Sci..

[38]  Thomas Sauerwald,et al.  Balls-into-bins with nearly optimal load distribution , 2013, SPAA.

[39]  E. Todeva Networks , 2007 .

[40]  Christoph Lenzen,et al.  Towards Optimal Synchronous Counting , 2015, PODC.

[41]  Alessandro Panconesi,et al.  Concentration of Measure for the Analysis of Randomized Algorithms , 2009 .

[42]  François Pottier,et al.  Information Flow , 2020, Encyclopedia of Creativity, Invention, Innovation and Entrepreneurship.

[43]  Gerald S. Wilkinson,et al.  Information transfer at evening bat colonies , 1992, Animal Behaviour.

[44]  T. Murata,et al.  Advanced modularity-specialized label propagation algorithm for detecting communities in networks , 2009, 0910.1154.

[45]  Fred B. Chambers,et al.  Distributed Computing , 2016, Lecture Notes in Computer Science.

[46]  David Peleg,et al.  Distributed Probabilistic Polling and Applications to Proportionate Agreement , 1999, ICALP.

[47]  Madhu Sudan,et al.  Limits of local algorithms over sparse random graphs , 2013, ITCS.

[48]  Amos Korman,et al.  Clock Synchronization and Estimation in Highly Dynamic Networks: An Information Theoretic Approach , 2015, SIROCCO.

[49]  Pierre Fraigniaud,et al.  Noisy rumor spreading and plurality consensus , 2015, Distributed Computing.

[50]  Andrea E. F. Clementi,et al.  Distributed community detection in dynamic graphs , 2013, Theor. Comput. Sci..

[51]  Bernard Chazelle,et al.  Natural algorithms , 2009, SODA.

[52]  Leslie Lamport,et al.  Reaching Agreement in the Presence of Faults , 1980, JACM.

[53]  D. Sumpter,et al.  Consensus Decision Making by Fish , 2008, Current Biology.

[54]  M. Barber,et al.  Detecting network communities by propagating labels under constraints. , 2009, Physical review. E, Statistical, nonlinear, and soft matter physics.

[55]  Devavrat Shah,et al.  Gossip Algorithms , 2009, Found. Trends Netw..

[56]  Gennaro Cordasco,et al.  Label propagation algorithm: a semi-synchronous approach , 2012, Int. J. Soc. Netw. Min..

[57]  Berthold Vöcking,et al.  Balanced allocations: the heavily loaded case , 2000, STOC '00.

[58]  Martin Raab,et al.  "Balls into Bins" - A Simple and Tight Analysis , 1998, RANDOM.

[59]  David Williams,et al.  Probability with Martingales , 1991, Cambridge mathematical textbooks.

[60]  R. Matthews,et al.  Ants. , 1898, Science.

[61]  David Kempe,et al.  A decentralized algorithm for spectral analysis , 2004, STOC '04.

[62]  Thomas Sauerwald,et al.  On the runtime and robustness of randomized broadcasting , 2006, Theor. Comput. Sci..

[63]  Réka Albert,et al.  Near linear time algorithm to detect community structures in large-scale networks. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[64]  Micah Adler,et al.  Parallel randomized load balancing , 1995, STOC '95.

[65]  Hilbert J. Kappen,et al.  Sufficient Conditions for Convergence of the Sum–Product Algorithm , 2005, IEEE Transactions on Information Theory.

[66]  Eli Upfal,et al.  Load Balancing in Arbitrary Network Topologies with Stochastic Adversarial Input , 2005, SIAM J. Comput..

[67]  Andrea E. F. Clementi,et al.  Self-Stabilizing Repeated Balls-into-Bins , 2015, SPAA.

[68]  Eli Upfal,et al.  Probability and Computing: Randomized Algorithms and Probabilistic Analysis , 2005 .

[69]  Devavrat Shah,et al.  Load balancing with memory , 2002, The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings..

[70]  Leslie Ann Goldberg,et al.  The natural work-stealing algorithm is stable , 2001, Proceedings 2001 IEEE International Conference on Cluster Computing.

[71]  Peter L. Hammer,et al.  Discrete Applied Mathematics , 1993 .

[72]  Leslie Lamport,et al.  Time, clocks, and the ordering of events in a distributed system , 1978, CACM.

[73]  Danny Dolev,et al.  Fast self-stabilizing byzantine tolerant digital clock synchronization , 2008, PODC '08.

[74]  Ravi B. Boppana,et al.  Eigenvalues and graph bisection: An average-case analysis , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[75]  David Doty,et al.  Timing in chemical reaction networks , 2013, SODA.

[76]  Christoph Lenzen,et al.  Efficient Counting with Optimal Resilience , 2015, DISC.

[77]  Milan Vojnovic,et al.  Using Three States for Binary Consensus on Complete Graphs , 2009, IEEE INFOCOM 2009.

[78]  Yair Weiss,et al.  Correctness of Local Probability Propagation in Graphical Models with Loops , 2000, Neural Computation.

[79]  Horst Trinker,et al.  Brief Announcement: Rapid Asynchronous Plurality Consensus , 2017, PODC.

[80]  B. Hajek Hitting-time and occupation-time bounds implied by drift analysis with applications , 1982, Advances in Applied Probability.

[81]  Merav Parter,et al.  A Polylogarithmic Gossip Algorithm for Plurality Consensus , 2016, PODC.

[82]  Scott Shenker,et al.  Epidemic algorithms for replicated database maintenance , 1988, OPSR.

[83]  N. Linial,et al.  Expander Graphs and their Applications , 2006 .

[84]  Mahmoud Fouz,et al.  Asymptotically Optimal Randomized Rumor Spreading , 2010, Electron. Notes Discret. Math..

[85]  Danny Dolev,et al.  On Self-stabilizing Synchronous Actions Despite Byzantine Attacks , 2007, DISC.

[86]  David Soloveichik,et al.  Stable leader election in population protocols requires linear time , 2015, Distributed Computing.

[87]  Mark Jerrum,et al.  The Metropolis Algorithm for Graph Bisection , 1998, Discret. Appl. Math..

[88]  Johannes Gehrke,et al.  Gossip-based computation of aggregate information , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[89]  John N. Tsitsiklis,et al.  Problems in decentralized decision making and computation , 1984 .

[90]  Ho-Lin Chen,et al.  Speed faults in computation by chemical reaction networks , 2014, Distributed Computing.

[91]  Cristopher Moore,et al.  Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[92]  Michael J. Fischer,et al.  Stabilizing Consensus in Mobile Networks , 2006, DCOSS.

[93]  Martin E. Dyer,et al.  The Solution of Some Random NP-Hard Problems in Polynomial Expected Time , 1989, J. Algorithms.

[94]  Michael O. Rabin,et al.  Randomized byzantine generals , 1983, 24th Annual Symposium on Foundations of Computer Science (sfcs 1983).

[95]  B. Pittel On spreading a rumor , 1987 .

[96]  Jukka Suomela,et al.  Survey of local algorithms , 2013, CSUR.

[97]  Eli Upfal,et al.  Balanced Allocations , 1999, SIAM J. Comput..

[98]  Bernhard Haeupler,et al.  Breathe before speaking: efficient information dissemination despite noisy, limited and anonymous communication , 2013, Distributed Computing.

[99]  Mohammed Abdullah,et al.  Global majority consensus by local majority polling on graphs , 2012, 2014 7th International Conference on NETwork Games, COntrol and OPtimization (NetGCoop).

[100]  Anand D. Sarwate,et al.  Broadcast Gossip Algorithms for Consensus , 2009, IEEE Transactions on Signal Processing.

[101]  Andrea E. F. Clementi,et al.  Flooding Time of Edge-Markovian Evolving Graphs , 2010, SIAM J. Discret. Math..

[102]  Roger Wattenhofer,et al.  Stone Age Distributed Computing , 2012, ArXiv.

[103]  Dan Alistarh,et al.  Polylogarithmic-Time Leader Election in Population Protocols , 2015, ICALP.

[104]  Moez Draief,et al.  Convergence Speed of Binary Interval Consensus , 2010, 2010 Proceedings IEEE INFOCOM.

[105]  Desh Ranjan,et al.  Balls and bins: A study in negative dependence , 1996, Random Struct. Algorithms.

[106]  Laurent Massoulié,et al.  Non-backtracking Spectrum of Random Graphs: Community Detection and Non-regular Ramanujan Graphs , 2014, 2015 IEEE 56th Annual Symposium on Foundations of Computer Science.

[107]  Luca Trevisan,et al.  Find Your Place: Simple Distributed Algorithms for Community Detection , 2015, SODA.

[108]  Laurent Massoulié,et al.  Community detection thresholds and the weak Ramanujan property , 2013, STOC.

[109]  Christian Scheideler,et al.  Towards a scalable and robust DHT , 2006, SPAA.

[110]  Elchanan Mossel,et al.  A Proof of the Block Model Threshold Conjecture , 2013, Combinatorica.

[111]  Lars Nagel,et al.  Self-stabilizing Balls & Bins in Batches , 2016, ArXiv.

[112]  Silvio Lattanzi,et al.  Rumor spreading in social networks , 2009, Theor. Comput. Sci..

[113]  M. Degroot Reaching a Consensus , 1974 .

[114]  H. Robbins A Remark on Stirling’s Formula , 1955 .

[115]  Ioana Dumitriu,et al.  Recovery and Rigidity in a Regular Stochastic Block Model , 2016, SODA.

[116]  Kathryn B. Laskey,et al.  Stochastic blockmodels: First steps , 1983 .

[117]  F. Chung Laplacians of graphs and Cheeger inequalities , 1993 .

[118]  Noga Alon,et al.  A Biological Solution to a Fundamental Distributed Computing Problem , 2011, Science.

[119]  I. Couzin,et al.  Effective leadership and decision-making in animal groups on the move , 2005, Nature.

[120]  Martin Vetterli,et al.  Interval consensus: From quantized gossip to voting , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[121]  James Aspnes,et al.  An Introduction to Population Protocols , 2007, Bull. EATCS.

[122]  W. Kahan,et al.  The Rotation of Eigenvectors by a Perturbation. III , 1970 .

[123]  Jennifer L. Welch,et al.  Self-Stabilizing Clock Synchronization in the Presence of ByzantineFaults ( Preliminary Version ) Shlomi Dolevy , 1995 .

[124]  Christian Scheideler,et al.  Towards a Scalable and Robust DHT , 2006, SPAA '06.

[125]  Eamonn B. Mallon,et al.  Information flow, opinion polling and collective intelligence in house-hunting social insects. , 2002, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[126]  M. Newman Spread of epidemic disease on networks. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[127]  Dan Alistarh,et al.  Fast and Exact Majority in Population Protocols , 2015, PODC.

[128]  Leslie Lamport Solved problems, unsolved problems and non-problems in concurrency , 1985, OPSR.

[129]  John N. Tsitsiklis,et al.  Convergence Speed in Distributed Consensus and Averaging , 2009, SIAM J. Control. Optim..

[130]  Shay Kutten,et al.  A self-stabilizing transformer for population protocols with covering , 2011, Theor. Comput. Sci..

[131]  David S. Johnson,et al.  Some Simplified NP-Complete Graph Problems , 1976, Theor. Comput. Sci..

[132]  Stephen P. Boyd,et al.  Distributed average consensus with least-mean-square deviation , 2007, J. Parallel Distributed Comput..

[133]  Elchanan Mossel,et al.  Majority dynamics and aggregation of information in social networks , 2012, Autonomous Agents and Multi-Agent Systems.

[134]  Emmanuel Abbe,et al.  Exact Recovery in the Stochastic Block Model , 2014, IEEE Transactions on Information Theory.

[135]  Masafumi Yamashita,et al.  Fair Circulation of a Token , 2002, IEEE Trans. Parallel Distributed Syst..

[136]  J. van Leeuwen,et al.  Theoretical Computer Science , 2003, Lecture Notes in Computer Science.

[137]  Sriram V. Pemmaraju,et al.  On the Analysis of a Label Propagation Algorithm for Community Detection , 2012, ICDCN.

[138]  Thomas M. Cover,et al.  Network Information Theory , 2001 .

[139]  George Giakkoupis,et al.  Efficient Plurality Consensus, Or: the Benefits of Cleaning up from Time to Time , 2016, ICALP.

[140]  Jurek Czyzowicz,et al.  Efficient Information Exchange in the Random Phone-Call Model , 2010, ICALP.

[141]  David Eisenstat,et al.  Stably computable predicates are semilinear , 2006, PODC '06.

[142]  Petar Maymounkov,et al.  Global computation in a poorly connected world: fast rumor spreading with no dependence on conductance , 2011, STOC '12.

[143]  V. N. Bogaevski,et al.  Matrix Perturbation Theory , 1991 .

[144]  David Peleg,et al.  Local majorities, coalitions and monopolies in graphs: a review , 2002, Theor. Comput. Sci..

[145]  Elchanan Mossel,et al.  Spectral redemption in clustering sparse networks , 2013, Proceedings of the National Academy of Sciences.

[146]  Can M. Le,et al.  Concentration and regularization of random graphs , 2015, Random Struct. Algorithms.

[147]  Colin Cooper,et al.  Random Walks, Interacting Particles, Dynamic Networks: Randomness Can Be Helpful , 2011, SIROCCO.

[148]  C. Lanczos An iteration method for the solution of the eigenvalue problem of linear differential and integral operators , 1950 .

[149]  Amos Korman,et al.  Minimizing message size in stochastic communication patterns: fast self-stabilizing protocols with 3 bits , 2016, Distributed Computing.

[150]  Emal Pasarly Time , 2011, Encyclopedia of Evolutionary Psychological Science.

[151]  Pierre Fraigniaud,et al.  Local Distributed Decision , 2011, 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science.

[152]  Amos Israeli,et al.  Token management schemes and random walks yield self-stabilizing mutual exclusion , 1990, PODC '90.

[153]  Jean-Pierre Eckmann,et al.  Desert ants achieve reliable recruitment across noisy interactions , 2013, Journal of The Royal Society Interface.

[154]  James Aspnes,et al.  Faster randomized consensus with an oblivious adversary , 2012, PODC '12.

[155]  Christoph Lenzen,et al.  Clock Synchronization: Open Problems in Theory and Practice , 2009, SOFSEM.

[156]  Seif Haridi,et al.  Distributed Algorithms , 1992, Lecture Notes in Computer Science.

[157]  Christoph Lenzen,et al.  Tight bounds for clock synchronization , 2010, JACM.

[158]  C. McDiarmid Concentration , 1862, The Dental register.

[159]  R. D. Harkness,et al.  Central place foraging by an ant (Cataglyphis bicolor Fab.): a model of searching , 1985, Animal Behaviour.

[160]  Ted Herman Phase Clocks for Transient Fault Repair , 2000, IEEE Trans. Parallel Distributed Syst..

[161]  Stephen Wolfram,et al.  A New Kind of Science , 2003, Artificial Life.

[162]  Elchanan Mossel,et al.  Reaching Consensus on Social Networks , 2010, ICS.

[163]  J. M. Harrison,et al.  Brownian Models of Feedforward Queueing Networks: Quasireversibility and Product Form Solutions , 1992 .

[164]  Colin Cooper,et al.  The Power of Two Choices in Distributed Voting , 2014, ICALP.

[165]  L. Goddard Information Theory , 1962, Nature.

[166]  Mehmet E. Yildiz,et al.  Binary Opinion Dynamics with Stubborn Agents , 2013, TEAC.

[167]  Noga Alon,et al.  Many random walks are faster than one , 2007, SPAA '08.

[168]  Elchanan Mossel,et al.  Reconstruction and estimation in the planted partition model , 2012, Probability Theory and Related Fields.

[169]  Michael W. Sneddon,et al.  Single-cell quantification of IL-2 response by effector and regulatory T cells reveals critical plasticity in immune response , 2010, Molecular systems biology.

[170]  Emanuele Natale,et al.  On the Voting Time of the Deterministic Majority Process , 2015, MFCS.

[171]  Chandler Davis The rotation of eigenvectors by a perturbation , 1963 .

[172]  Stephen P. Boyd,et al.  Randomized gossip algorithms , 2006, IEEE Transactions on Information Theory.

[173]  Richard K. Belew,et al.  No Two-State CA for Density Classification Exists , 1994 .

[174]  E. Wilson,et al.  The Superorganism: The Beauty, Elegance, and Strangeness of Insect Societies , 2008 .

[175]  Charles Bordenave,et al.  A new proof of Friedman's second eigenvalue theorem and its extension to random lifts , 2015, Annales scientifiques de l'École normale supérieure.

[176]  B. Bassler,et al.  Quorum sensing: cell-to-cell communication in bacteria. , 2005, Annual review of cell and developmental biology.

[177]  Sheldon M. Ross,et al.  Stochastic Processes , 2018, Gauge Integral Structures for Stochastic Calculus and Quantum Electrodynamics.

[178]  Dana Randall,et al.  A Markov Chain Algorithm for Compression in Self-Organizing Particle Systems , 2016, PODC.

[179]  Paul G. Spirakis,et al.  Determining majority in networks with local interactions and very small local memory , 2014, Distributed Computing.

[180]  Friedhelm Meyer auf der Heide,et al.  Efficient PRAM simulation on a distributed memory machine , 1992, STOC '92.

[181]  M. Carroll,et al.  The complement system in regulation of adaptive immunity , 2004, Nature Immunology.

[182]  Andrea E. F. Clementi,et al.  Plurality Consensus in the Gossip Model , 2014, SODA.

[183]  Robert Elsässer,et al.  On the Influence of Graph Density on Randomized Gossiping , 2015, 2015 IEEE International Parallel and Distributed Processing Symposium.

[184]  Nicola Santoro,et al.  Design and analysis of distributed algorithms , 2006, Wiley series on parallel and distributed computing.

[185]  Anisur Rahaman Molla,et al.  Near-optimal random walk sampling in distributed networks , 2012, 2012 Proceedings IEEE INFOCOM.

[186]  Luca Trevisan,et al.  Simple dynamics for plurality consensus , 2013, Distributed Computing.

[187]  Christian Scheideler,et al.  Amoebot - a new model for programmable matter , 2014, SPAA.

[188]  Shlomi Dolev,et al.  Possible and Impossible Self-Stabilizing Digital Clock Synchronization in General Graphs , 1997, Real-Time Systems.

[189]  Shay Kutten,et al.  Time Optimal Synchronous Self Stabilizing Spanning Tree , 2013, DISC.