FastCast: A Throughput- and Latency-Efficient Total Order Broadcast Protocol

Many uniform total order broadcast protocols have been designed in the last 30 years. Unfortunately, none of them achieves both optimal throughput and low latency. Indeed, protocols achieving optimal throughput rely on a ring dissemination pattern, which induces high latencies. Protocols achieving low latency rely on IP multicast and fail to achieve good throughput because of message losses. In this paper, we describe FastCast, the first protocol that achieves both optimal throughput and low latency. To achieve low latency, FastCast relies on IP multicast. To achieve optimal throughput, FastCast defines a protocol responsible for dynamically computing the throughput at which processes can send IP multicast messages. Thanks to this dynamic bandwidth allocation protocol, FastCast allows multiple processes to simultaneously send messages, while avoiding message losses. An evaluation of FastCast on a cluster of 8 machines shows that it indeed achieves optimal throughput and a very low latency.

[1]  Fred B. Schneider,et al.  Implementing fault-tolerant services using the state machine approach: a tutorial , 1990, CSUR.

[2]  Sam Toueg,et al.  Unreliable failure detectors for reliable distributed systems , 1996, JACM.

[3]  Nancy A. Lynch,et al.  Distributed Algorithms , 1992, Lecture Notes in Computer Science.

[4]  Alan O. Freier,et al.  Multicast Transport Protocol , 1992, RFC.

[5]  André Schiper,et al.  A hierarchy of totally ordered multicasts , 1995, Proceedings. 14th Symposium on Reliable Distributed Systems.

[6]  Roy Friedman,et al.  Packing messages as a tool for boosting the performance of total ordering protocols , 1997, Proceedings. The Sixth IEEE International Symposium on High Performance Distributed Computing (Cat. No.97TB100183).

[7]  Kenneth P. Birman,et al.  Reliable communication in the presence of failures , 1987, TOCS.

[8]  Virgil D. Gligor,et al.  A Fault-Tolerant Protocol for Atomic Broadcast , 1990, IEEE Trans. Parallel Distributed Syst..

[9]  Péter Urbán,et al.  Token-based atomic broadcast using unreliable failure detectors , 2004, Proceedings of the 23rd IEEE International Symposium on Reliable Distributed Systems, 2004..

[10]  Jongsung Kim,et al.  A total ordering protocol using a dynamic token-passing scheme , 1997, Distributed Syst. Eng..

[11]  Achour Mostéfaoui,et al.  Consensus-Based Fault-Tolerant Total Order Multicast , 2001, IEEE Trans. Parallel Distributed Syst..

[12]  Todd Montgomery,et al.  A High Performance Totally Ordered Multicast Protocol , 1994, Dagstuhl Seminar on Distributed Systems.

[13]  Flaviu Cristian New asynchronous atomic broadcast protocols , 1990, [1990] Proceedings. Workshop on the Management of Replicated Data.

[14]  William H. Sanders,et al.  Numerical performability evaluation of a group multicast protocol , 1996, Distributed Syst. Eng..

[15]  Emmanuelle Anceaume,et al.  A lightweight solution to uniform atomic broadcast for asynchronous systems , 1997, Proceedings of IEEE 27th International Symposium on Fault Tolerant Computing.

[16]  Sam Toueg,et al.  Fault-tolerant broadcasts and related problems , 1993 .

[17]  Newtop: a fault-tolerant group communication protocol , 1995, Proceedings of 15th International Conference on Distributed Computing Systems.

[18]  Vivek Agrawala,et al.  Asynchronous Fault-Tolerant Total Ordering Algorithms , 1993, SIAM J. Comput..

[19]  Hector Garcia-Molina,et al.  Ordered and reliable multicast communication , 1991, TOCS.

[20]  André Schiper,et al.  Theory and Practice in Distributed Systems , 1995, Lecture Notes in Computer Science.

[21]  Michael B. Jones,et al.  FUSE: Lightweight Guaranteed Distributed Failure Notification , 2004, OSDI.

[22]  Rachid Guerraoui,et al.  High Throughput Total Order Broadcast for Cluster Environments , 2006, International Conference on Dependable Systems and Networks (DSN'06).

[23]  Sam Toueg,et al.  Reliable Broadcast in Synchronous and Asynchronous Environments (Preliminary Version) , 1989, WDAG.

[24]  Andrew S. Tanenbaum,et al.  An evaluation of the Amoeba group communication system , 1996, Proceedings of 16th International Conference on Distributed Computing Systems.

[25]  Rachid Guerraoui,et al.  Throughput optimal total order broadcast for cluster environments , 2010, TOCS.

[26]  Flaviu Cristian,et al.  High-performance asynchronous atomic broadcast , 1997, Distributed Syst. Eng..

[27]  Fernando Pedone,et al.  Ring Paxos: A high-throughput atomic broadcast protocol , 2010, 2010 IEEE/IFIP International Conference on Dependable Systems & Networks (DSN).

[28]  Tony P. Ng Ordered broadcasts for large applications , 1991, [1991] Proceedings Tenth Symposium on Reliable Distributed Systems.

[29]  Robbert van Renesse,et al.  Reliable Distributed Computing with the Isis Toolkit , 1994 .

[30]  Richard D. Schlichting,et al.  Preserving and using context information in interprocess communication , 1989, TOCS.

[31]  Louise E. Moser,et al.  The Totem single-ring ordering and membership protocol , 1995, TOCS.

[32]  Jo-Mei Chang,et al.  Reliable broadcast protocols , 1984, TOCS.

[33]  Jean-Yves Le Boudec,et al.  Rate adaptation, Congestion Control and Fairness: A Tutorial , 2000 .

[34]  Claudiu Danilov,et al.  The Spread Toolkit: Architecture and Performance , 2004 .

[35]  Kenneth P. Birman,et al.  Exploiting virtual synchrony in distributed systems , 1987, SOSP '87.