Performing work with asynchronous processors: Message-delay-sensitive bounds

This paper considers the problem of performing tasks in asynchronous distributed settings. This problem, called Do-All, has been substantially studied in synchronous models, but there is a dearth of efficient algorithms for asynchronous message-passing processors. Do-All can be trivially solved without any communication by an algorithm where each processor performs all tasks. Assuming p processors and t tasks, this requires work @Q(p.t). Thus, it is important to develop subquadratic solutions (when p and t are comparable) by trading computation for communication. Following the observation that it is not possible to obtain subquadratic work when the message delay d is substantial, e.g., [email protected](t), this work pursues a message-delay-sensitive approach. Here, the upper bounds on work and communication are given as functions of p, t, and d, the upper bound on message delays, however, algorithms have no knowledge of d and they cannot rely on the existence of an upper bound on d. This paper presents two families of asynchronous algorithms achieving, for the first time, subquadratic work as long as d=o(t). The first family uses as its basis a shared-memory algorithm without having to emulate atomic registers assumed by that algorithm. These deterministic algorithms have work O(tp^@[email protected]?t/[email protected]?^@e) for any @e>0. The second family uses specific permutations of tasks, with certain combinatorial properties, to sequence the work of the processors. These randomized (deterministic) algorithms have expected (worst-case) work O(tlogp+pdlog(2+t/d)). Another important contribution in this work is the first delay-sensitive lower bound for this problem that helps explain the behavior of our algorithms: any randomized (deterministic) algorithm has expected (worst-case) work of @W(t+pdlog"d"+"1t).

[1]  Nancy A. Lynch,et al.  RAMBO: A Reconfigurable Atomic Memory Service for Dynamic Networks , 2002, DISC.

[2]  Chryssis Georgiou,et al.  Efficient Gossip and Robust Distributed Computation , 2003, DISC.

[3]  Donald E. Knuth,et al.  The art of computer programming, volume 3: (2nd ed.) sorting and searching , 1998 .

[4]  Z. M. Kedem,et al.  Combining tentative and definite executions for dependable parallel computing , 1990 .

[5]  Alexander Russell,et al.  The Complexity of Distributed Cooperation in the Presence of Failures , 2000, OPODIS.

[6]  Dariusz R. Kowalski,et al.  Emulating Shared-Memory Do-All Algorithms in Asynchronous Message-Passing Systems , 2003, OPODIS.

[7]  Z. M. Kedem,et al.  Combining tentative and definite executions for very fast dependable parallel computing , 1991, STOC '91.

[8]  Noga Alon,et al.  The Probabilistic Method , 2015, Fundamentals of Ramsey Theory.

[9]  Donald Ervin Knuth,et al.  The Art of Computer Programming , 1968 .

[10]  Joseph Naor,et al.  Constructions of Permutation Arrays for Certain Scheduling Cost Measures , 1995, Random Struct. Algorithms.

[11]  Andrzej Lingas,et al.  The do-all problem in broadcast networks , 2001, PODC '01.

[12]  Moti Yung,et al.  Time-optimal message-efficient work performance in the presence of faults , 1994, PODC '94.

[13]  Nancy A. Lynch,et al.  Consensus in the presence of partial synchrony , 1988, JACM.

[14]  Joseph Y. Halpern,et al.  Performing Work Efficiently in the Presence of Faults , 1998, SIAM J. Comput..

[15]  Bogdan S. Chlebus,et al.  Performing tasks on synchronous restartable message-passing processors , 2001, Distributed Computing.

[16]  Richard J. Anderson,et al.  Algorithms for the Certified Write-All Problem , 1997, SIAM J. Comput..

[17]  Moti Yung,et al.  Resolving message complexity of Byzantine Agreement and beyond , 1995, Proceedings of IEEE 36th Annual Foundations of Computer Science.

[18]  Alexander A. Shvartsman,et al.  Fault-Tolerant Parallel Computation , 1997 .

[19]  Dariusz R. Kowalski,et al.  Bounding Work and Communication in Robust Cooperative Computation , 2002, DISC.

[20]  Prabhakar Ragde,et al.  Parallel Algorithms with Processor Failures and Delays , 1996, J. Algorithms.