The overhead of consensus failure recovery

Many reliable distributed systems are consensus-based and typically operate under two modes: a fast normal mode in failure-free synchronous periods, and a slower recovery mode following asynchrony and failures. A lot of work has been devoted to optimize the normal mode, but little has focused on optimizing the recovery mode. This paper seeks to understand whether the recovery mode is inherently slower than the normal mode. In particular, we consider consensus algorithms in the round-based eventually synchronous model of [11], where t out of n processes may fail by crashing, messages may be lost, and the system may be asynchronous for arbitrarily long, but eventually the system becomes synchronous and no new failure occurs (we say that the system becomes stable). For t   ≥   n/3, we prove a lower bound of three rounds for achieving a global decision whenever the system becomes stable, and we contrast this with a bound of two rounds when t  <  n/3. We then give matching algorithms for both t   ≥   n/3 and t   <   n/3.

[1]  Rachid Guerraoui,et al.  Fast Indulgent Consensus with Zero Degradation , 2002, EDCC.

[2]  Yair Amir,et al.  From total order to database replication , 2002, Proceedings 22nd International Conference on Distributed Computing Systems.

[3]  Robbert van Renesse,et al.  Reliable Distributed Computing with the Isis Toolkit , 1994 .

[4]  Leslie Lamport,et al.  The part-time parliament , 1998, TOCS.

[5]  Flaviu Cristian,et al.  An efficient, fault-tolerant protocol for replicated data management , 1985, PODS '85.

[6]  Danny Dolev,et al.  Early stopping in Byzantine agreement , 1990, JACM.

[7]  Rachid Guerraoui.  Revistiting the Relationship Between Non-Blocking Atomic Commitment and Consensus , 1995, WDAG.

[8]  Flaviu Cristian,et al.  The Timed Asynchronous Distributed System Model , 1999, IEEE Trans. Parallel Distributed Syst..

[9]  Russell Greiner,et al.  Efficient reasoning , 2001, CSUR.

[10]  Idit Keidar,et al.  On the Cost of Fault-Tolerant Consensus When There Are No Faults - A Tutorial , 2003, LADC.

[11]  David Powell,et al.  Group communication , 1996, CACM.

[12]  Idit Keidar,et al.  Efficient message ordering in dynamic networks , 1996, PODC '96.

[13]  Achour Mostéfaoui,et al.  Leader-Based Consensus , 2001, Parallel Process. Lett..

[14]  Nancy A. Lynch,et al.  Impossibility of distributed consensus with one faulty process , 1985, JACM.

[15]  Idit Keidar,et al.  Group communication specifications: a comprehensive study , 2001, CSUR.

[16]  Sam Toueg,et al.  Unreliable failure detectors for reliable distributed systems , 1996, JACM.

[17]  Barbara Liskov,et al.  Viewstamped Replication: A General Primary Copy , 1988, PODC.

[18]  Chandramohan A. Thekkath,et al.  Frangipani: a scalable distributed file system , 1997, SOSP.

[19]  Eli Gafni,et al.  Round-by-Round Fault Detectors: Unifying Synchrony and Asynchrony (Extended Abstract). , 1998, PODC 1998.

[20]  Nancy A. Lynch,et al.  Consensus in the presence of partial synchrony , 1988, JACM.

[21]  Butler W. Lampson,et al.  How to Build a Highly Available System Using Consensus , 1996, WDAG.

[22]  André Schiper,et al.  Uniform consensus is harder than consensus , 2004, J. Algorithms.

[23]  Nancy A. Lynch,et al.  A Lower Bound for the Time to Assure Interactive Consistency , 1982, Inf. Process. Lett..

[24]  Leslie Lamport,et al.  Time, clocks, and the ordering of events in a distributed system , 1978, CACM.

[25]  Roy Friedman,et al.  Fast replicated state machines over partitionable networks , 1997, Proceedings of SRDS'97: 16th IEEE Symposium on Reliable Distributed Systems.

[26]  Idit Keidar,et al.  A simple proof of the uniform consensus synchronous lower bound , 2003, Inf. Process. Lett..

[27]  Nicola Santoro,et al.  Time is Not a Healer , 1989, STACS.

[28]  Fred B. Schneider,et al.  Implementing fault-tolerant services using the state machine approach: a tutorial , 1990, CSUR.

[29]  Leslie Lamport,et al.  The Byzantine Generals Problem , 1982, TOPL.

[30]  Rachid Guerraoui,et al.  The inherent price of indulgence , 2002, PODC '02.