Total ordering algorithms

1 I n t r o d u c t i o n We consider the problem of placing a total order on messages in an asynchronous distributed system that is subject to processor and communication faults. The total ordering problem requires that nonfaulty processors construct identical total orders of messages and that all processors construct consistent total orders. The total ordering problem is a remarkably difficult problem in a fault-tolerant asynchronous distributed system; in fact, we can show that in such a system it is impossible to construct a total ordering algorithm that is guaranteed to terminate. Thus, we consider algorithms whose probability of terminating increases asymptotically to unity as a processor takes more steps. There are basically two ways to incorporate probabilities into the possible executions of an algorithm. In the first approach [1], randomization is introduced into the algorithm. The second approach, and the one we adopt, assumes probabilistic behavior of the communication mechanism. The Total algorithms presented here are fault-tolerant algorithms for placing a total order on messages whose probability of determining an extension to the total order increases asymptotically to unity as the number of steps taken by a processor tends to infinity. The algorithms start with a partial order on messages that is provided by the Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Association for Computing Machinery. To copy otherwise, or to republish, requires a fee and/or specific permission. © 1991 ACM 089791-382-5/91/0003/0375 $1.50 underlying communication mechanism and incrementally converts the partial order into a total order in such a way that every nonfaulty processor constructs the same total order, even though some of the processors may not have received some of the messages. Given a total ordering of messages, distributed operations requiring consensus decisions can be carried out using simple sequential algorithms. The placement of a total order on messages provides an alternative approach to solving the consensus problem [3,5,11,14]. Fischer, Lynch and Paterson {8]~ have shown that, for an asynchronous distributed system with even just one faulty processor, it is impossible to construct a consensus algorithm that is guaranteed to terminate. For synchronous and partially synchronous systems, fault-tolerant algorithms for reaching consensus have been developed [6,7,9,16], but synchronous systems are limited in flexibility. For asynchronous systems, asymptotic algorithms have been devised for which the probability of reaching consensus increases asymptotically to uni ty [1,4,13,15], but those algorithms involve the exchange of many messages. We know of no other fault-tolerant algorithms like the ones presented here for placing a total order on messages. The partial order on which the Total algorithms are based can easily be constructed from almost any underlying communication mechanism. However, when the Total algorithms are implemented using the Trans broadcast protocol [12], remarkably high efliciencies can be achieved. Efficiency relies on the narrowness of the partial order; when a broadcast communication mechanism is used, the partial order is typically very narrow. In this case, each decision to extend the total order requires only one broadcast message plus any retransmissions that may be required because messages are not received immediately. An advantage of the Total algorithms is that they place a total order on messages without the need to broadcast extra messages. This paper is organized as follows. In Section 2 we define our model of computation. Next, in Section 3 we introduce the basic idea of the Total algorithms, and in Sections 4 and 5 we present Total algorithms that are resilient to fewer than n /3 and n /2 fail-stop processor faults. Then, in Section 6 we state the theorems which establish partial correctness and probabilistic termination of the algorithms, and in Section 7 we compare the complexity of the two algorithms. Finally, in Section 8 we present the impossibility result.