Agreement with Satoshi - On the Formalization of Nakamoto Consensus

The term Nakamoto consensus is generally used to refer to Bitcoin’s novel consensus mechanism, by which agreement on its underlying transaction ledger is reached. It is argued that this agreement protocol represents the core innovation behind Bitcoin, because it promises to facilitate the decentralization of trusted third parties. Specifically, Nakamoto consensus seeks to enable mutually distrusting entities with weak pseudonymous identities to reach eventual agreement while the set of participants may change over time. When the Bitcoin white paper was published in late 2008, it lacked a formal analysis of the protocol and the guarantees it claimed to provide. It would take the scientific community several years before first steps towards such a formalization of the Bitcoin protocol and Nakamoto consensus were presented. However, since then the number of works addressing this topic has grown substantially, providing many new and valuable insights. Herein, we present a coherent picture of advancements towards the formalization of Nakamoto consensus, as well as a contextualization in respect to previous research on the agreement problem and fault tolerant distributed computing. Thereby, we outline how Bitcoin’s consensus mechanism sets itself apart from previous approaches and where it can provide new impulses and directions to the scientific community. Understanding the core properties and characteristics of Nakamoto consensus is of key importance, not only for assessing the security and reliability of various blockchain systems that are based on the fundamentals of this scheme, but also for designing future systems that aim to fulfill comparable goals. I. THE IMPORTANCE OF NAKAMOTO CONSENSUS The recent explosive increase in both economic valuation and technical interest towards Bitcoin, cryptocurrencies, and distributed ledgers in general, is mirrored by equally growing research efforts from the scientific community to better understand, employ and extend upon the fundamental principles that govern these technologies. Not only has the body of peer-reviewed literature directly related to Bitcoin and cryptocurrencies increased substantially as outlined by this recent taxonomy [XWS17], but a lot of work is also presented online in the form of pre-prints, e-prints, and informally as blog posts, following the publishing spirit of the original Bitcoin white paper [Nak08a]. Moreover, many other research fields are also exploring how the underlying concepts behind blockchain technologies could be applied in their domain. This renders it difficult for both researchers and practitioners to get a coherent picture of the state-of-the-art in this emerging field. We therefore 1One of the earliest uses of the term Nakamoto consensus can be attributed to a blog post by Nick Szabo in [Sza14], after which it appeared in scientific publications such as [BMC+15], [LTKS15]. believe that further systematization efforts related to Bitcoin and blockchain technologies, following the comprehensive overview of research perspectives and challenges for Bitcoin presented by Bonneau et al. in 2015 [BMC15], are necessary. In particular, the study and formalization of the Bitcoin protocol and its underlying Nakamoto consensus has seen significant advances in recent years (e.g. [KP15], [GKL16], [BPS16a]) that are not yet systematically exposed. Recent work provides a broad overview of different consensus mechanisms in the context of blockchain technologies [BSAB17], however we feel that a more in-depth analysis of the relationship between Nakamoto consensus and previous approaches to Byzantine consensus is still outstanding. We hereby narrow this gap by relating research towards Nakamoto consensus to other key insights and aspects on the topic of consensus. Consensus is a fundamental building block in fault tolerant and distributed computing, and the guarantees a consensus protocol provides can greatly impact the overall security and reliability of (distributed ledger) systems [CV17]. Bitcoin promises to solve the double spending problem in a distributed, peer-to-peer environment without the necessity to rely on a trusted third party by enabling participants to reach (eventual) agreement on the state changes to a shared transaction ledger. Nakamoto consensus hence lies at the core of this system. Without an in-depth understanding of this mechanism, entire categories of newly designed systems, as well as the applications that are built on top of them, are potentially vulnerable to attacks [NG16], [CV17]. Modifications to consensus related rules, even if they appear small or straightforward, can fundamentally impact underlying incentives and greatly affect security guarantees [ZP17]. As the ecosystem around “blockchain“ has grown into a multi-billion dollar industry, severe failures could have far-reaching consequences and a long-term negative impact on the entire field. On the other hand, fundamental insights on Nakamoto consensus have already spawned novel and hybrid consensus approaches that exhibit interesting properties and characteristics, while providing the necessary frameworks to analyze and evaluate the correctness and security of such approaches. Combining aspects of “classical“ Byzantine fault tolerant (BFT) consensus protocols with Nakamoto consensus may help to address increasing concerns regarding the scalability and performance of blockchain technologies. Motivated by this emerging new area of research, we set out to paint a coherent picture of the insights and findings that have been presented on the topic of Nakamoto consensus and the fundamental mechanisms behind Bitcoin and similar blockchain protocols.

[1]  J. Coolidge The Gambler's Ruin , 1909 .

[2]  Leslie Lamport,et al.  Reaching Agreement in the Presence of Faults , 1980, JACM.

[3]  Danny Dolev,et al.  Unanimity in an unknown and unreliable environment , 1981, 22nd Annual Symposium on Foundations of Computer Science (sfcs 1981).

[4]  Nancy A. Lynch,et al.  A Lower Bound for the Time to Assure Interactive Consistency , 1982, Inf. Process. Lett..

[5]  Leslie Lamport,et al.  The Byzantine Generals Problem , 1982, TOPL.

[6]  Michael Ben-Or,et al.  Another advantage of free choice (Extended Abstract): Completely asynchronous agreement protocols , 1983, PODC '83.

[7]  Michael J. Fischer,et al.  The Consensus Problem in Unreliable Distributed Systems (A Brief Survey) , 1983, FCT.

[8]  Michael O. Rabin,et al.  Randomized byzantine generals , 1983, 24th Annual Symposium on Foundations of Computer Science (sfcs 1983).

[9]  Danny Dolev,et al.  On the minimal synchronism needed for distributed consensus , 1983, 24th Annual Symposium on Foundations of Computer Science (sfcs 1983).

[10]  Leslie Lamport,et al.  Using Time Instead of Timeout for Fault-Tolerant Distributed Systems. , 1984, TOPL.

[11]  Nancy A. Lynch,et al.  Impossibility of distributed consensus with one faulty process , 1985, JACM.

[12]  Nancy A. Lynch,et al.  Reaching approximate agreement in the presence of faults , 1986, JACM.

[13]  Kenneth P. Birman,et al.  Exploiting virtual synchrony in distributed systems , 1987, SOSP '87.

[14]  Nancy A. Lynch,et al.  Consensus in the presence of partial synchrony , 1988, JACM.

[15]  Amos Fiat,et al.  Untraceable Electronic Cash , 1990, CRYPTO.

[16]  Maurice Herlihy,et al.  Impossibility and universality results for wait-free synchronization , 1988, PODC '88.

[17]  Fred B. Schneider,et al.  Implementing fault-tolerant services using the state machine approach: a tutorial , 1990, CSUR.

[18]  Maurice Herlihy,et al.  Wait-free synchronization , 1991, TOPL.

[19]  Moni Naor,et al.  Pricing via Processing or Combatting Junk Mail , 1992, CRYPTO.

[20]  Aleta Marie Ricciardi,et al.  The Group Membership Problem in Asynchronous Systems , 1993 .

[21]  Nancy A. Lynch,et al.  Distributed Algorithms , 1992, Lecture Notes in Computer Science.

[22]  Gil Neiger,et al.  Distributed Consensus Revisited , 1994, Inf. Process. Lett..

[23]  Sam Toueg,et al.  A Modular Approach to Fault-Tolerant Broadcasts and Related Problems , 1994 .

[24]  A secure group membership protocol , 1994, IEEE Symposium on Security and Privacy.

[25]  Ronald L. Rivest,et al.  Time-lock Puzzles and Timed-release Crypto , 1996 .

[26]  Michael K. Reiter A Secure Group Membership Protocol , 1996, IEEE Trans. Software Eng..

[27]  Sam Toueg,et al.  Unreliable failure detectors for reliable distributed systems , 1996, JACM.

[28]  Michael K. Reiter,et al.  Unreliable intrusion detection in distributed computations , 1997, Proceedings 10th Computer Security Foundations Workshop.

[29]  Bela Ban Design and Implementation of a Reliable Group Communication Toolkit for Java , 1998 .

[30]  Leslie Lamport,et al.  The part-time parliament , 1998, TOCS.

[31]  Miguel Oom Temudo de Castro,et al.  Practical Byzantine fault tolerance , 1999, OSDI '99.

[32]  John G. Brainard,et al.  Client Puzzles: A Cryptographic Countermeasure Against Connection Depletion Attacks , 1999, NDSS.

[33]  Achour Mostéfaoui,et al.  From Binary Consensus to Multivalued Consensus in asynchronous message-passing systems , 2000, Inf. Process. Lett..

[34]  Ran Canetti,et al.  Security and Composition of Multiparty Cryptographic Protocols , 2000, Journal of Cryptology.

[35]  Eric A. Brewer,et al.  Towards robust distributed systems (abstract) , 2000, PODC '00.

[36]  Luís E. T. Rodrigues,et al.  Appia, a flexible protocol kernel supporting multiple coordinated channels , 2001, Proceedings 21st International Conference on Distributed Computing Systems.

[37]  Ran Canetti,et al.  Universally composable security: a new paradigm for cryptographic protocols , 2001, Proceedings 2001 IEEE International Conference on Cluster Computing.

[38]  Louise E. Moser,et al.  The SecureRing group communication system , 2001, TSEC.

[39]  John R. Douceur,et al.  The Sybil Attack , 2002, IPTPS.

[40]  Miguel Castro,et al.  Practical byzantine fault tolerance and proactive recovery , 2002, TOCS.

[41]  Rachid Guerraoui,et al.  Encapsulating Failure Detection: From Crash to Byzantine Failures , 2002, Ada-Europe.

[42]  Nancy A. Lynch,et al.  Brewer's conjecture and the feasibility of consistent, available, partition-tolerant web services , 2002, SIGA.

[43]  Louise E. Moser,et al.  Byzantine Fault Detectors for Solving Consensus , 2003, Comput. J..

[44]  Michael K. Reiter,et al.  Objects shared by Byzantine processes , 2000, Distributed Computing.

[45]  Paulo Veríssimo Uncertainty and predictability: can they be reconciled? , 2003 .

[46]  A. Schiper,et al.  Total order broadcast and multicast algorithms: Taxonomy and survey , 2003, CSUR.

[47]  Miguel Correia,et al.  How to tolerate half less one Byzantine nodes in practical distributed systems , 2004, Proceedings of the 23rd IEEE International Symposium on Reliable Distributed Systems, 2004..

[48]  Victor Shoup,et al.  Random Oracles in Constantinople: Practical Asynchronous Byzantine Agreement Using Cryptography , 2000, Journal of Cryptology.

[49]  Flaviu Cristian,et al.  Reaching agreement on processor-group membrship in synchronous distributed systems , 1991, Distributed Computing.

[50]  J. Aspnes,et al.  Exposing Computationally-Challenged Byzantine Impostors , 2005 .

[51]  Michael Dahlin,et al.  BAR fault tolerance for cooperative services , 2005, SOSP '05.

[52]  Michael Dahlin,et al.  BAR gossip , 2006, OSDI '06.

[53]  Brian Neil Levine,et al.  A Survey of Solutions to the Sybil Attack , 2006 .

[54]  Michael J. Fischer,et al.  Stabilizing Consensus in Mobile Networks , 2006, DCOSS.

[55]  Yair Amir,et al.  The Spread Wide Area Group Communication System , 2007 .

[56]  Michel Raynal,et al.  Looking for a Definition of Dynamic Distributed Systems , 2007, PaCT.

[57]  Robert Griesemer,et al.  Paxos made live: an engineering perspective , 2007, PODC '07.

[58]  Jaap-Henk Hoepman,et al.  Distributed Double Spending Prevention , 2007, Security Protocols Workshop.

[59]  Alysson Neves Bessani,et al.  Byzantine Consensus with Unknown Participants , 2008, OPODIS.

[60]  Rachele Fuzzati,et al.  A formal approach to fault tolerant distributed consensus , 2008 .

[61]  R. Tempo,et al.  Las Vegas randomized algorithms in distributed consensus problems , 2008, 2008 American Control Conference.

[62]  M. Raynal,et al.  The Price of Anonymity: Optimal Consensus Despite Asynchrony, Crash and Anonymity , 2009, DISC.

[63]  Michael Dahlin,et al.  Making Byzantine Fault Tolerant Systems Tolerate Byzantine Faults , 2009, NSDI.

[64]  Satoshi Nakamoto Bitcoin : A Peer-to-Peer Electronic Cash System , 2009 .

[65]  Miguel Correia,et al.  Asynchronous Byzantine consensus with 2f+1 processes , 2010, SAC '10.

[66]  Miguel Correia,et al.  Byzantine consensus in asynchronous message-passing systems: a survey , 2011, Int. J. Crit. Comput. Based Syst..

[67]  Rachid Guerraoui,et al.  Introduction to Reliable and Secure Distributed Programming , 2011 .

[68]  Ueli Maurer,et al.  Universally Composable Synchronous Computation , 2013, TCC.

[69]  Jonathan Katz,et al.  Byzantine Agreement with a Rational Adversary , 2012, ICALP.

[70]  Aviv Zohar,et al.  Accelerating Bitcoin's Transaction Processing. Fast Money Grows on Trees, Not Chains , 2013, IACR Cryptol. ePrint Arch..

[71]  Miguel Correia,et al.  Efficient Byzantine Fault-Tolerance , 2013, IEEE Transactions on Computers.

[72]  Christian Decker,et al.  Information propagation in the Bitcoin network , 2013, IEEE P2P 2013 Proceedings.

[73]  Joseph J. LaViola,et al.  Byzantine Consensus from Moderately-Hard Puzzles : A Model for Bitcoin , 2014 .

[74]  Emin Gün Sirer,et al.  Majority Is Not Enough: Bitcoin Mining Is Vulnerable , 2013, Financial Cryptography.

[75]  Meni Rosenfeld,et al.  Analysis of Hashrate-Based Double Spending , 2014, ArXiv.

[76]  Aggelos Kiayias,et al.  Speed-Security Tradeoffs in Blockchain Protocols , 2015, IACR Cryptol. ePrint Arch..

[77]  Yoad Lewenberg,et al.  Inclusive Block Chain Protocols , 2015, Financial Cryptography.

[78]  Ethan Heilman,et al.  Eclipse Attacks on Bitcoin's Peer-to-Peer Network , 2015, USENIX Security Symposium.

[79]  Jason Teutsch,et al.  Demystifying Incentives in the Consensus Computer , 2015, CCS.

[80]  Jeremy Clark,et al.  SoK: Research Perspectives and Challenges for Bitcoin and Cryptocurrencies , 2015, 2015 IEEE Symposium on Security and Privacy.

[81]  Aggelos Kiayias,et al.  The Bitcoin Backbone Protocol: Analysis and Applications , 2015, EUROCRYPT.

[82]  Marko Vukolic,et al.  The Next 700 BFT Protocols , 2015, ACM Trans. Comput. Syst..

[83]  Hubert Ritzdorf,et al.  Tampering with the Delivery of Blocks and Transactions in Bitcoin , 2015, IACR Cryptol. ePrint Arch..

[84]  Marko Vukolic,et al.  The Quest for Scalable Blockchain Fabric: Proof-of-Work vs. BFT Replication , 2015, iNetSeC.

[85]  Roger Wattenhofer,et al.  Byzantine Agreement with Median Validity , 2015, OPODIS.

[86]  Abhi Shelat,et al.  Analysis of the Blockchain Protocol in Asynchronous Networks , 2017, EUROCRYPT.

[87]  Vincent Gramoli,et al.  The Blockchain Anomaly , 2016, 2016 IEEE 15th International Symposium on Network Computing and Applications (NCA).

[88]  Elaine Shi,et al.  FruitChains: A Fair Blockchain , 2017, IACR Cryptol. ePrint Arch..

[89]  Sooyong Park,et al.  Where Is Current Research on Blockchain Technology?—A Systematic Review , 2016, PloS one.

[90]  Aviv Zohar,et al.  Optimal Selfish Mining Strategies in Bitcoin , 2015, Financial Cryptography.

[91]  Kartik Nayak,et al.  Stubborn Mining: Generalizing Selfish Mining and Combining with an Eclipse Attack , 2016, 2016 IEEE European Symposium on Security and Privacy (EuroS&P).

[92]  Aggelos Kiayias,et al.  Ouroboros: A Provably Secure Proof-of-Stake Blockchain Protocol , 2017, CRYPTO.

[93]  Silvio Micali,et al.  ALGORAND: The Efficient and Democratic Ledger , 2016, ArXiv.

[94]  Yoad Lewenberg,et al.  SPECTRE: A Fast and Scalable Cryptocurrency Protocol , 2016, IACR Cryptol. ePrint Arch..

[95]  Corentin Travers,et al.  Anonymity-Preserving Failure Detectors , 2016, DISC.

[96]  Aviv Zohar,et al.  Bitcoin's Security Model Revisited , 2016, ArXiv.

[97]  Hubert Ritzdorf,et al.  On the Security and Performance of Proof of Work Blockchains , 2016, IACR Cryptol. ePrint Arch..

[98]  Björn Scheuermann,et al.  Bitcoin and Beyond: A Technical Survey on Decentralized Digital Currencies , 2016, IEEE Communications Surveys & Tutorials.

[99]  Kartik Nayak,et al.  Solidus: An Incentive-compatible Cryptocurrency Based on Permissionless Byzantine Consensus , 2016, ArXiv.

[100]  Elaine Shi,et al.  Snow White: Provably Secure Proofs of Stake , 2016, IACR Cryptol. ePrint Arch..

[101]  Emin Gün Sirer,et al.  Bitcoin-NG: A Scalable Blockchain Protocol , 2015, NSDI.

[102]  Elaine Shi,et al.  The Honey Badger of BFT Protocols , 2016, CCS.

[103]  R. Pass Hybrid Consensus : Scalable Permissionless Consensus , 2016 .

[104]  Cesare Pautasso,et al.  A Taxonomy of Blockchain-Based Systems for Architecture Design , 2017, 2017 IEEE International Conference on Software Architecture (ICSA).

[105]  Elaine Shi,et al.  The Sleepy Model of Consensus , 2017, ASIACRYPT.

[106]  Marko Vukolic,et al.  Blockchain Consensus Protocols in the Wild , 2017, DISC.

[107]  Aggelos Kiayias,et al.  Proofs of Work for Blockchain Protocols , 2017, IACR Cryptol. ePrint Arch..

[108]  Elaine Shi,et al.  Thunderella: Blockchains with Optimistic Instant Confirmation , 2018, IACR Cryptol. ePrint Arch..

[109]  Silvio Micali,et al.  Algorand: Scaling Byzantine Agreements for Cryptocurrencies , 2017, IACR Cryptol. ePrint Arch..

[110]  Aggelos Kiayias,et al.  Ouroboros Praos: An adaptively-secure, semi-synchronous proof-of-stake protocol , 2017, IACR Cryptol. ePrint Arch..

[111]  Bart Preneel,et al.  On the Necessity of a Prescribed Block Validity Consensus: Analyzing Bitcoin Unlimited Mining Protocol , 2017, CoNEXT.

[112]  Aggelos Kiayias,et al.  The Bitcoin Backbone Protocol with Chains of Variable Difficulty , 2017, CRYPTO.

[113]  George Danezis,et al.  Consensus in the Age of Blockchains , 2017, ArXiv.