Randomized Load Balancing on Networks with Stochastic Inputs

Iterative load balancing algorithms for indivisible tokens have been studied intensively in the past. Complementing previous worst-case analyses, we study an average-case scenario where the load inputs are drawn from a fixed probability distribution. For cycles, tori, hypercubes and expanders, we obtain almost matching upper and lower bounds on the discrepancy, the difference between the maximum and the minimum load. Our bounds hold for a variety of probability distributions including the uniform and binomial distribution but also distributions with unbounded range such as the Poisson and geometric distribution. For graphs with slow convergence like cycles and tori, our results demonstrate a substantial difference between the convergence in the worst- and average-case. An important ingredient in our analysis is new upper bound on the t-step transition probability of a general Markov chain, which is derived by invoking the evolving set process.

[1]  V. Climenhaga.  Markov chains and mixing times , 2013 .

[2]  Peter Sanders.  Analysis of nearest neighbor load balancing algorithms for random loads , 1999, Parallel Comput..

[3]  Thomas Sauerwald,et al.  Randomized Load Balancing on Networks with Stochastic Inputs , 2017, International Colloquium on Automata, Languages and Programming.

[4]  Thorsten Gerber,et al.  Handbook Of Mathematical Functions , 2016 .

[5]  A. C. Berry.  The accuracy of the Gaussian approximation to the sum of independent variates , 1941 .

[6]  W. Hoeffding.  Probability Inequalities for sums of Bounded Random Variables , 1963 .

[7]  Jacques E. Boillat,et al.  Load Balancing and Poisson Equation in a Graph , 1990, Concurr. Pract. Exp..

[8]  Mark Jerrum,et al.  Approximate Counting, Uniform Generation and Rapidly Mixing Markov Chains , 1987, WG.

[9]  Stephen P. Boyd,et al.  Randomized gossip algorithms , 2006, IEEE Transactions on Information Theory.

[10]  Uday S. Dixit,et al.  Finite Element Methods For Engineers , 2009 .

[11]  Thomas Sauerwald,et al.  Tight Bounds for Randomized Load Balancing on Arbitrary Network Topologies , 2012, 2012 IEEE 53rd Annual Symposium on Foundations of Computer Science.

[12]  Changjun Jiang,et al.  A fast adaptive load balancing method for parallel particle-based simulations , 2009, Simul. Model. Pract. Theory.

[13]  Aravind Srinivasan,et al.  Improved distributed algorithms for coloring and network decomposition problems , 1992, STOC '92.

[14]  Thomas Sauerwald,et al.  The impact of randomization in smoothing networks , 2008, PODC '08.

[15]  Isaac D. Scherson,et al.  An analysis of diffusive load-balancing , 1994, SPAA '94.

[16]  Dan Alistarh,et al.  Are lock-free concurrent algorithms practically wait-free? , 2013, STOC.

[17]  Arndt Bode.  Load balancing in distributed memory multiprocessors , 1991, [1991] Proceedings, Advanced Computer Technology, Reliable Systems and Applications.

[18]  Luca Trevisan,et al.  Find Your Place: Simple Distributed Algorithms for Community Detection , 2015, SODA.

[19]  Russell Lyons.  Asymptotic Enumeration of Spanning Trees , 2005, Comb. Probab. Comput..

[20]  Yuval Rabani,et al.  Local divergence of Markov chains and the analysis of iterative load-balancing schemes , 1998, Proceedings 39th Annual Symposium on Foundations of Computer Science (Cat. No.98CB36280).

[21]  Aravind Srinivasan,et al.  Randomized Distributed Edge Coloring via an Extension of the Chernoff-Hoeffding Bounds , 1997, SIAM J. Comput..

[22]  Prasad Tetali,et al.  Distributed Random Walks , 2013, JACM.

[23]  Bruce M. Maggs,et al.  Tight analyses of two local load balancing algorithms , 1995, STOC '95.

[24]  Roy D. Williams,et al.  Performance of dynamic load balancing algorithms for unstructured mesh calculations , 1991, Concurr. Pract. Exp..

[25]  S. Muthukrishnan,et al.  First- and Second-Order Diffusive Methods for Rapid, Coarse, Distributed Load Balancing , 1996, Theory of Computing Systems.

[26]  S. Muthukrishnan,et al.  Dynamic Load Balancing by Random Matchings , 1996, J. Comput. Syst. Sci..

[27]  Richard M. Karp,et al.  Load balancing in dynamic structured peer-to-peer systems , 2006, Perform. Evaluation.

[28]  M. Habib.  Probabilistic methods for algorithmic discrete mathematics , 1998 .

[29]  Noga Alon,et al.  The Probabilistic Method , 2015, Fundamentals of Ramsey Theory.

[30]  J. Pach,et al.  Wiley‐Interscience Series in Discrete Mathematics and Optimization , 2011 .

[31]  Eli Upfal,et al.  Load Balancing in Arbitrary Network Topologies with Stochastic Adversarial Input , 2005, SIAM J. Comput..

[32]  Ashish Goel,et al.  Stochastic load balancing and related problems , 1999, 40th Annual Symposium on Foundations of Computer Science (Cat. No.99CB37039).