Asynchronous Random Polling Dynamic Load Balancing

Many applications in parallel processing have to traverse large, implicitly defined trees with irregular shape. The receiver initiated load balancing algorithm random polling has long been known to be very efficient for these problems in practice. For any Ɛ > 0, we prove that its parallel execution time is at most (1+Ɛ)Tseq/P+O(Tatomic+h(1/Ɛ+Trout+ Tsplit)) with high probability, where Trout, Tsplit and Tatomic bound the time for sending a message, splitting a subproblem and finishing a small unsplittable subproblem respectively. The maximum splitting depth h is related to the depth of the computation tree. Previous work did not prove efficiency close to one and used less accurate models. In particular, our machine model allows asynchronous communication with nonconstant message delays and does not assume that communication takes place in rounds. This model is compatible with the LogP model.

[1]  Peter Sanders,et al.  A Scalable Parallel Tree Search Library , 1996 .

[2]  Ramesh Subramonian,et al.  LogP: towards a realistic model of parallel computation , 1993, PPOPP '93.

[3]  Paul G. Spirakis,et al.  Scheduling Algorithms for Strict Multithreaded Computations , 1996, ISAAC.

[4]  Michael Mitzenmacher,et al.  Analyses of Load Stealing Models Based on Families of Differential Equations , 2000, Theory of Computing Systems.

[5]  Peter Sanders,et al.  A detailed analysis of random polling dynamic load balancing , 1994, Proceedings of the International Symposium on Parallel Architectures, Algorithms and Networks (ISPAN).

[6]  Oliver Vornberger,et al.  Superlinear Speedup for Parallel Backtracking , 1987, ICS.

[7]  Vipin Kumar,et al.  Automatic test pattern generation on parallel processors , 1991, Parallel Comput..

[8]  Michael Mitzenmacher,et al.  Analyses of load stealing models based on differential equations , 1998, SPAA '98.

[9]  Volker Heun,et al.  Efficient Dynamic Embedding of Arbitrary Binary Trees into Hypercubes , 1996, IRREGULAR.

[10]  Jacques Chassin de Kergommeaux,et al.  Parallel logic programming systems , 1994, CSUR.

[11]  H. T. Kung,et al.  Communication complexity for parallel divide-and-conquer , 1991, [1991] Proceedings 32nd Annual Symposium of Foundations of Computer Science.

[12]  Katherine Yelick,et al.  Randomized load balancing for tree-structured computation , 1994, Proceedings of IEEE Scalable High Performance Computing Conference.

[13]  Vipin Kumar,et al.  Scalable Load Balancing Techniques for Parallel Computers , 1994, J. Parallel Distributed Comput..

[14]  G.S. Bloom,et al.  Applications of numbered undirected graphs , 1977, Proceedings of the IEEE.

[15]  R. Korf.  An Optimal Admissible Tree Search , 1985 .

[16]  Peter Sanders,et al.  Analysis of Random Polling Dynamic Load Balancing , 1997 .

[17]  Paolo Toth,et al.  Knapsack Problems: Algorithms and Computer Implementations , 1990 .

[18]  Roger Wattenhofer,et al.  An Inherent Bottleneck in Distributed Counting , 1998, J. Parallel Distributed Comput..

[19]  Robert D. Blumofe,et al.  Scheduling multithreaded computations by work stealing , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[20]  George Karypis,et al.  Introduction to Parallel Computing , 1994 .

[21]  Amnon Barak,et al.  An adaptive granularity control algorithm for the parallel execution of functional programs , 1993, Future Gener. Comput. Syst..

[22]  Burkhard Monien,et al.  Studying overheads in massively parallel MIN/MAX-tree evaluation , 1994, SPAA '94.

[23]  Peter Sanders.  Tree Shaped Computations as a Model for Parallel Applications , 1998 .

[24]  Paul G. Spirakis,et al.  A New Scheduling Algorithm for General Strict Multithreaded Computations , 1999, DISC.

[25]  C. Greg Plaxton,et al.  Thread Scheduling for Multiprogrammed Multiprocessors , 1998, SPAA '98.

[26]  Peter Sanders,et al.  Better Algorithms for Parallel Backtracking , 1995, IRREGULAR.

[27]  Peter Sanders.  Lastverteilungsalgorithmen für parallele Tiefensuche , 1997 .

[28]  Bruce M. Maggs,et al.  Randomized Routing and Sorting on Fixed-Connection Networks , 1994, J. Algorithms.

[29]  Udi Manber,et al.  DIB—a distributed implementation of backtracking , 1987, TOPL.

[30]  Wolfgang Ertel.  Parallele Suche mit randomisiertem Wettbewerb in Inferenzsystemen , 1993, DISKI.

[31]  Alexander Reinefeld,et al.  Scalability of Massively Parallel Depth-First Search , 1994 .

[32]  Ewald Speckenmeyer,et al.  A Fast Parallel Sat{solver | Eecient Workload Balancing a Fast Parallel Sat{solver | Eecient Workload Balancing , 1994 .

[33]  Christos Goumopoulos,et al.  Parallel Crew Scheduling on Workstation Networks Using PVM , 1997, PVM/MPI.