Distal: A framework for implementing fault-tolerant distributed algorithms

We introduce Distal, a new framework that simplifies turning pseudocode of fault tolerant distributed algorithms into efficient executable code. Without proper tool support, even small amounts of pseudocode normally ends up in several thousands of non-trivial lines of Java or C++. Distal is implemented as a library in Scala and consists of two main parts: a domain specific language (DSL) in which algorithms are expressed and an efficient messaging layer that deals with low level issues such as connection management, threading and (de)serialization. The DSL is designed such that implementations of distributed algorithms highly resemble the pseudocode found in research papers. By writing code that is close to the protocol description, one can be more convinced that the implemented system really reflects the protocol specification on paper. Distal does not only make it simple and intuitive to implement distributed algorithms but it also leads to efficient implementations.

[1]  Yair Amir,et al.  Paxos for System Builders , 2008 .

[2]  Marc Najork,et al.  Boxwood: Abstractions as the Foundation for Storage Infrastructure , 2004, OSDI.

[3]  Miguel Castro,et al.  Practical byzantine fault tolerance and proactive recovery , 2002, TOCS.

[4]  Brett D. Fleisch,et al.  The Chubby lock service for loosely-coupled distributed systems , 2006, OSDI '06.

[5]  Yair Amir,et al.  Paxos for System Builders: an overview , 2008, LADIS '08.

[6]  Atul Singh,et al.  Applying Prolog to develop distributed systems , 2010, Theory Pract. Log. Program..

[7]  Keith Marzullo,et al.  Mencius: Building Efficient Replicated State Machine for WANs , 2008, OSDI.

[8]  Joseph M. Hellerstein,et al.  I do declare: consensus in a logic language , 2010, OPSR.

[9]  Robbert van Renesse,et al.  Building adaptive systems using ensemble , 1998 .

[10]  Gene Tsudik,et al.  Secure spread: an integrated architecture for secure group communication , 2005, IEEE Transactions on Dependable and Secure Computing.

[11]  Amin Vahdat,et al.  Mace: language support for building distributed systems , 2007, PLDI '07.

[12]  Tushar Deepak Chandra,et al.  Paxos Made Live - An Engineering Perspective (2006 Invited Talk) , 2007 .

[13]  Leslie Lamport,et al.  The part-time parliament , 1998, TOCS.

[14]  Kenneth P. Birman,et al.  The ISIS project: real experience with a fault tolerant programming system , 1990, EW 4.

[15]  Nancy A. Lynch,et al.  IOA: A Language for Specifying, Programming, and Validating Distributed Systems Draft , 1997 .

[16]  Marko Vukolic,et al.  The Next 700 BFT Protocols , 2015, ACM Trans. Comput. Syst..

[17]  Nancy A. Lynch,et al.  Revisiting the PAXOS algorithm , 1997, Theor. Comput. Sci..

[18]  Michael Ben-Or,et al.  Another advantage of free choice (Extended Abstract): Completely asynchronous agreement protocols , 1983, PODC '83.

[19]  Mahadev Konar,et al.  ZooKeeper: Wait-free Coordination for Internet-scale Systems , 2010, USENIX ATC.

[20]  Robbert van Renesse,et al.  How the Hidden Hand Shapes the Market for Software Reliability , 2006 .

[21]  Robbert van Renesse,et al.  Building Adaptive Systems Using Ensemble , 1998, Softw. Pract. Exp..

[22]  André Schiper,et al.  S-Paxos: Offloading the Leader for High Throughput State Machine Replication , 2012, 2012 IEEE 31st Symposium on Reliable Distributed Systems.

[23]  Rachid Guerraoui,et al.  One Acceptor is Enough , 2010 .

[24]  Mark Bickford,et al.  ShadowDB: A Replicated Database on a Synthesized Consensus Core , 2012, HotDep.

[25]  Peng Li,et al.  Paxos Replicated State Machines as the Basis of a High-Performance Data Store , 2011, NSDI.

[26]  David E. Culler,et al.  SEDA: an architecture for well-conditioned, scalable internet services , 2001, SOSP.

[27]  Luiz André Barroso,et al.  The tail at scale , 2013, CACM.

[28]  Ramakrishna Kotla,et al.  Zyzzyva , 2007, SOSP.

[29]  Robert Griesemer,et al.  Paxos made live: an engineering perspective , 2007, PODC '07.

[30]  Nancy A. Lynch,et al.  An introduction to input/output automata , 1989 .