BAR fault tolerance for cooperative services

This paper describes a general approach to constructing cooperative services that span multiple administrative domains. In such environments, protocols must tolerate both Byzantine behaviors when broken, misconfigured, or malicious nodes arbitrarily deviate from their specification and rational behaviors when selfish nodes deviate from their specification to increase their local benefit. The paper makes three contributions: (1) It introduces the BAR (Byzantine, Altruistic, Rational) model as a foundation for reasoning about cooperative services; (2) It proposes a general three-level architecture to reduce the complexity of building services under the BAR model; and (3) It describes an implementation of BAR-B the first cooperative backup service to tolerate both Byzantine users and an unbounded number of rational users. At the core of BAR-B is an asynchronous replicated state machine that provides the customary safety and liveness guarantees despite nodes exhibiting both Byzantine and rational behaviors. Our prototype provides acceptable performance for our application: our BAR-tolerant state machine executes 15 requests per second, and our BAR-B backup service can back up 100MB of data in under 4 minutes.

[1]  J. Nash NON-COOPERATIVE GAMES , 1951, Classics in Game Theory.

[2]  A. Rapoport,et al.  The Game of Chicken , 1966 .

[3]  John C. Harsanyil A GENERAL THEORY OF RATIONAL BEHAVIOR IN GAME SITUATIONS , 1966 .

[4]  R. Aumann Subjectivity and Correlation in Randomized Strategies , 1974 .

[5]  Leslie Lamport,et al.  The Byzantine Generals Problem , 1982, TOPL.

[6]  Adi Shamir,et al.  A method for obtaining digital signatures and public-key cryptosystems , 1978, CACM.

[7]  Sam Toueg,et al.  Asynchronous consensus and broadcast protocols , 1985, JACM.

[8]  Kenneth P. Birman,et al.  Replication and fault-tolerance in the ISIS system , 1985, SOSP '85.

[9]  Nancy A. Lynch,et al.  Impossibility of distributed consensus with one faulty process , 1985, JACM.

[10]  Martín Abadi,et al.  A logic of authentication , 1990, TOCS.

[11]  Fred B. Schneider,et al.  Implementing fault-tolerant services using the state machine approach: a tutorial , 1990, CSUR.

[12]  Drew Fudenberg,et al.  Game theory (3. pr.) , 1991 .

[13]  Fred B. Schneider What good are models and what models are good , 1993 .

[14]  Yoram Moses,et al.  Fully polynomial Byzantine agreement in t + 1 rounds , 1993, STOC.

[15]  Michael K. Reiter,et al.  The Rampart Toolkit for Building High-Integrity Services , 1994, Dagstuhl Seminar on Distributed Systems.

[16]  Fred B. Schneider,et al.  Hypervisor-based fault tolerance , 1995, TOCS.

[17]  Luigi Rizzo,et al.  Effective erasure codes for reliable computer communication protocols , 1997, CCRV.

[18]  Michael K. Reiter,et al.  Byzantine quorum systems , 1997, STOC '97.

[19]  Michael K. Reiter,et al.  Secure and scalable replication in Phalanx , 1998, Proceedings Seventeenth IEEE Symposium on Reliable Distributed Systems (Cat. No.98CB36281).

[20]  G. Mailath Do People Play Nash Equilibrium? Lessons From Evolutionary Game Theory , 1998 .

[21]  Yoram Moses,et al.  Fully Polynomial Byzantine Agreement for n > 3t Processors in t + 1 Rounds , 1998, SIAM J. Comput..

[22]  Leslie Lamport,et al.  The part-time parliament , 1998, TOCS.

[23]  Avinash Dixit,et al.  Games of Strategy , 1999 .

[24]  Stefan Savage,et al.  TCP congestion control with a misbehaving receiver , 1999, CCRV.

[25]  Eytan Adar,et al.  Free Riding on Gnutella , 2000, First Monday.

[26]  Victoria Ungureanu,et al.  Law-governed interaction: a coordination and control mechanism for heterogeneous distributed systems , 2000, TSEM.

[27]  Christos H. Papadimitriou,et al.  Algorithms, Games, and the Internet , 2001, ICALP.

[28]  Noam Nisan,et al.  Algorithmic Mechanism Design , 2001, Games Econ. Behav..

[29]  Antony I. T. Rowstron,et al.  Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility , 2001, SOSP.

[30]  Robert Axelrod,et al.  The Evolution of Strategies in the Iterated Prisoner's Dilemma , 2001 .

[31]  Joan Feigenbaum,et al.  Sharing the Cost of Multicast Transmissions , 2001, J. Comput. Syst. Sci..

[32]  Miguel Castro,et al.  Using abstraction to improve fault tolerance , 2001, Proceedings Eighth Workshop on Hot Topics in Operating Systems.

[33]  Arun Venkataramani,et al.  Proceedings of the 5th Symposium on Operating Systems Design and Implementation Tcp Nice: a Mechanism for Background Transfers , 2022 .

[34]  John R. Douceur,et al.  The Sybil Attack , 2002, IPTPS.

[35]  Miguel Castro,et al.  Practical byzantine fault tolerance and proactive recovery , 2002, TOCS.

[36]  Mike Hibler,et al.  An integrated experimental environment for distributed systems and networks , 2002, OSDI '02.

[37]  Joan Feigenbaum,et al.  Distributed algorithmic mechanism design: recent results and future directions , 2002, DIALM '02.

[38]  K. Eliaz Fault Tolerant Implementation , 2002 .

[39]  Brian D. Noble,et al.  Proceedings of the 5th Symposium on Operating Systems Design and Implementation Pastiche: Making Backup Cheap and Easy , 2022 .

[40]  Jacob R. Lorch,et al.  Farsite: federated, available, and reliable storage for an incompletely trusted environment , 2002, OSDI '02.

[41]  Srinivasan Seshan,et al.  Selfish behavior and stability of the internet: a game-theoretic analysis of TCP , 2002, SIGCOMM.

[42]  Krishna P. Gummadi,et al.  Measurement, modeling, and analysis of a peer-to-peer file-sharing workload , 2003, SOSP '03.

[43]  Amin Vahdat,et al.  SHARP: an architecture for secure resource peering , 2003, SOSP '03.

[44]  Miguel Castro,et al.  BASE: Using abstraction to improve fault tolerance , 2003, TOCS.

[45]  Arun Venkataramani,et al.  Separating agreement from execution for byzantine fault tolerant services , 2003, SOSP '03.

[46]  Brian D. Noble,et al.  Samsara: honor among thieves in peer-to-peer storage , 2003, SOSP '03.

[47]  Mary Baker,et al.  Preserving peer replicas by rate-limited sampled voting , 2003, SOSP '03.

[48]  Dan S. Wallach,et al.  Enforcing Fair Sharing of Peer-to-Peer Resources , 2003, IPTPS.

[49]  B. Cohen,et al.  Incentives Build Robustness in Bit-Torrent , 2003 .

[50]  Michael Burrows,et al.  A Cooperative Internet Backup Scheme , 2003, USENIX Annual Technical Conference, General Track.

[51]  Ben Y. Zhao,et al.  Pond: The OceanStore Prototype , 2003, FAST.

[52]  Vikram Srinivasan,et al.  Cooperation in wireless ad hoc networks , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[53]  David C. Parkes,et al.  Rationality and Self-Interest in Peer to Peer Networks , 2003, IPTPS.

[54]  Ben Y. Zhao,et al.  Awarded Best Student Paper! - Pond: The OceanStore Prototype , 2003 .

[55]  Peter Druschel,et al.  Incentives-Compatible Peer-to-Peer Multicast , 2004 .

[56]  Laurent Massoulié,et al.  Faithfulness in internet algorithms , 2004, PINS '04.

[57]  Peter Triantafillou,et al.  AESOP: Altruism-Endowed Self-organizing Peers , 2004, DBISP2P.

[58]  Joan Feigenbaum,et al.  Mechanism design for policy routing , 2004, PODC '04.

[59]  David C. Parkes,et al.  Specification faithfulness in networks with rational nodes , 2004, PODC '04.

[60]  Dan S. Wallach,et al.  A Taxonomy of Rational Attacks , 2005, IPTPS.

[61]  Ratul Mahajan,et al.  Sustaining cooperation in multi-hop wireless networks , 2005, NSDI.

[62]  Christos H. Papadimitriou,et al.  Free-riding and whitewashing in peer-to-peer systems , 2004, IEEE Journal on Selected Areas in Communications.

[63]  Christopher Batten,et al.  pStore: A Secure Peer-to-Peer Backup System∗ , 2007 .