SoK: Consensus in the Age of Blockchains

The core technical component of blockchains is consensus: how to reach agreement among a distributed network of nodes. A plethora of blockchain consensus protocols have been proposed---ranging from new designs, to novel modifications and extensions of consensus protocols from the classical distributed systems literature. The inherent complexity of consensus protocols and their rapid and dramatic evolution makes it hard to contextualize the design landscape. We address this challenge by conducting a systematization of knowledge of blockchain consensus protocols. After first discussing key themes in classical consensus protocols, we describe: (i) protocols based on proof-of-work; (ii) proof-of-X protocols that replace proof-of-work with more energy-efficient alternatives; and (iii) hybrid protocols that are compositions or variations of classical consensus protocols. This survey is guided by a systematization framework we develop, to highlight the various building blocks of blockchain consensus design, along with a discussion on their security and performance properties. We identify research gaps and insights for the community to consider in future research endeavours.

[1]  H. London The Meaning of Decentralization , 1975 .

[2]  Jim Gray,et al.  Notes on Data Base Operating Systems , 1978, Advanced Course: Operating Systems.

[3]  Dale Skeen,et al.  Nonblocking commit protocols , 1981, SIGMOD '81.

[4]  Leslie Lamport,et al.  The Byzantine Generals Problem , 1982, TOPL.

[5]  T. Maugh Why buy when you can rent? , 1984, Science.

[6]  Nancy A. Lynch,et al.  Impossibility of distributed consensus with one faulty process , 1985, JACM.

[7]  Nancy A. Lynch,et al.  Consensus in the presence of partial synchrony , 1988, JACM.

[8]  Fred B. Schneider,et al.  Implementing fault-tolerant services using the state machine approach: a tutorial , 1990, CSUR.

[9]  Moni Naor,et al.  Pricing via Processing or Combatting Junk Mail , 1992, CRYPTO.

[10]  Flaviu Cristian,et al.  Atomic Broadcast: From Simple Message Diffusion to Byzantine Agreement , 1995, Inf. Comput..

[11]  Leslie Lamport,et al.  The part-time parliament , 1998, TOCS.

[12]  Barbara Liskov,et al.  Viewstamped Replication: A New Primary Copy Method to Support Highly-Available Distributed Systems , 1999, PODC '88.

[13]  Miguel Oom Temudo de Castro,et al.  Practical Byzantine fault tolerance , 1999, OSDI '99.

[14]  C. Cachin,et al.  Random oracles in constantipole: practical asynchronous Byzantine agreement using cryptography (extended abstract) , 2000, PODC '00.

[15]  John R. Douceur,et al.  The Sybil Attack , 2002, IPTPS.

[16]  David Mazières,et al.  Kademlia: A Peer-to-Peer Information System Based on the XOR Metric , 2002, IPTPS.

[17]  Arun Venkataramani,et al.  Separating agreement from execution for byzantine fault tolerant services , 2003, SOSP '03.

[18]  B. Cohen,et al.  Incentives Build Robustness in Bit-Torrent , 2003 .

[19]  Ramakrishna Kotla,et al.  High throughput Byzantine fault tolerance , 2004, International Conference on Dependable Systems and Networks, 2004.

[20]  Michael Dahlin,et al.  BAR fault tolerance for cooperative services , 2005, SOSP '05.

[21]  Leslie Lamport,et al.  Consensus on transaction commit , 2004, TODS.

[22]  Candelaria Hernández-Goya,et al.  A rational approach to cryptographic protocols , 2007, Math. Comput. Model..

[23]  Atul Singh,et al.  BFT Protocols Under Fire , 2008, NSDI.

[24]  Jason Flinn,et al.  Tolerating Latency in Replicated State Machines Through Client Speculation , 2009, NSDI.

[25]  Christian Scheideler,et al.  Robust random number generation for peer-to-peer systems , 2006, Theor. Comput. Sci..

[26]  Satoshi Nakamoto Bitcoin : A Peer-to-Peer Electronic Cash System , 2009 .

[27]  Jens Groth,et al.  Short Pairing-Based Non-interactive Zero-Knowledge Arguments , 2010, ASIACRYPT.

[28]  Ramakrishna Kotla,et al.  Zyzzyva: speculative byzantine fault tolerance , 2007, TOCS.

[29]  Mahadev Konar,et al.  ZooKeeper: Wait-free Coordination for Internet-scale Systems , 2010, USENIX ATC.

[30]  John Lane,et al.  Prime: Byzantine Replication under Attack , 2011, IEEE Transactions on Dependable and Secure Computing.

[31]  Rachid Guerraoui,et al.  Introduction to Reliable and Secure Distributed Programming , 2011 .

[32]  Ivan Beschastnikh,et al.  Scalable consistency in Scatter , 2011, SOSP.

[33]  Flavio Paiva Junqueira,et al.  Zab: High-performance broadcast for primary-backup systems , 2011, 2011 IEEE/IFIP 41st International Conference on Dependable Systems & Networks (DSN).

[34]  B. Laurie An Efficient Distributed Currency , 2011 .

[35]  Aviv Zohar,et al.  On bitcoin and red balloons , 2012, EC '12.

[36]  Silvio Lattanzi,et al.  SoK: The Evolution of Sybil Defense via Social Networks , 2013, 2013 IEEE Symposium on Security and Privacy.

[37]  Aviv Zohar,et al.  Accelerating Bitcoin's Transaction Processing. Fast Money Grows on Trees, Not Chains , 2013, IACR Cryptol. ePrint Arch..

[38]  Christian Decker,et al.  Information propagation in the Bitcoin network , 2013, IEEE P2P 2013 Proceedings.

[39]  Stefan Savage,et al.  A fistful of bitcoins: characterizing payments among men with no names , 2013, Internet Measurement Conference.

[40]  Joshua A. Kroll,et al.  The Economics of Bitcoin Mining, or Bitcoin in the Presence of Adversaries , 2013 .

[41]  Ueli Maurer,et al.  Rational Protocol Design: Cryptography against Incentive-Driven Adversaries , 2013, 2013 IEEE 54th Annual Symposium on Foundations of Computer Science.

[42]  Brian F. Cooper Spanner: Google's globally-distributed database , 2013, SYSTOR '13.

[43]  Alysson Neves Bessani,et al.  State Machine Replication for the Masses with BFT-SMART , 2014, 2014 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks.

[44]  Eli Ben-Sasson,et al.  Zerocash: Decentralized Anonymous Payments from Bitcoin , 2014, 2014 IEEE Symposium on Security and Privacy.

[45]  Iddo Bentov,et al.  Proof of Activity: Extending Bitcoin's Proof of Work via Proof of Stake [Extended Abstract]y , 2014, PERV.

[46]  John K. Ousterhout,et al.  In Search of an Understandable Consensus Algorithm , 2014, USENIX ATC.

[47]  Emin Gün Sirer,et al.  Majority Is Not Enough: Bitcoin Mining Is Vulnerable , 2013, Financial Cryptography.

[48]  David Schwartz,et al.  The Ripple Protocol Consensus Algorithm , 2014 .

[49]  Daniel Davis Wood,et al.  ETHEREUM: A SECURE DECENTRALISED GENERALISED TRANSACTION LEDGER , 2014 .

[50]  Jae Kwon,et al.  Tendermint : Consensus without Mining , 2014 .

[51]  Elaine Shi,et al.  Permacoin: Repurposing Bitcoin Work for Data Preservation , 2014, 2014 IEEE Symposium on Security and Privacy.

[52]  Sergio Demian Lerner,et al.  DECOR + HOP : A Scalable Blockchain Protocol , 2015 .

[53]  Elaine Shi,et al.  Nonoutsourceable Scratch-Off Puzzles to Discourage Bitcoin Mining Coalitions , 2015, CCS.

[54]  David Mazières The Stellar Consensus Protocol: A Federated Model for Internet-level Consensus , 2015 .

[55]  Jason Teutsch,et al.  Demystifying Incentives in the Consensus Computer , 2015, CCS.

[56]  Ittay Eyal,et al.  The Miner's Dilemma , 2014, 2015 IEEE Symposium on Security and Privacy.

[57]  Aviv Zohar,et al.  Secure High-Rate Transaction Processing in Bitcoin , 2015, Financial Cryptography.

[58]  Jeremy Clark,et al.  SoK: Research Perspectives and Challenges for Bitcoin and Cryptocurrencies , 2015, 2015 IEEE Symposium on Security and Privacy.

[59]  Aggelos Kiayias,et al.  The Bitcoin Backbone Protocol: Analysis and Applications , 2015, EUROCRYPT.

[60]  Marko Vukolic,et al.  The Next 700 BFT Protocols , 2015, ACM Trans. Comput. Syst..

[61]  Marko Vukolic,et al.  The Quest for Scalable Blockchain Fabric: Proof-of-Work vs. BFT Replication , 2015, iNetSeC.

[62]  Marko Vukolic,et al.  XFT: Practical Fault Tolerance beyond Crashes , 2015, OSDI.

[63]  Marko Vukolic,et al.  Non-determinism in Byzantine Fault-Tolerant Replication , 2016, OPODIS.

[64]  Ethan Buchman,et al.  Tendermint: Byzantine Fault Tolerance in the Age of Blockchains , 2016 .

[65]  Alexander Chepurnoy Interactive Proof-of-stake , 2016, ArXiv.

[66]  George Danezis,et al.  Centrally Banked Cryptocurrencies , 2015, NDSS.

[67]  Elaine Shi,et al.  FruitChains: A Fair Blockchain , 2017, IACR Cryptol. ePrint Arch..

[68]  Sooyong Park,et al.  Where Is Current Research on Blockchain Technology?—A Systematic Review , 2016, PloS one.

[69]  Aviv Zohar,et al.  Optimal Selfish Mining Strategies in Bitcoin , 2015, Financial Cryptography.

[70]  Kartik Nayak,et al.  Stubborn Mining: Generalizing Selfish Mining and Combining with an Eclipse Attack , 2016, 2016 IEEE European Symposium on Security and Privacy (EuroS&P).

[71]  Aggelos Kiayias,et al.  Ouroboros: A Provably Secure Proof-of-Stake Blockchain Protocol , 2017, CRYPTO.

[72]  Fernando Pedone,et al.  Dynamic Scalable State Machine Replication , 2016, 2016 46th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN).

[73]  S. Matthew Weinberg,et al.  On the Instability of Bitcoin Without the Block Reward , 2016, CCS.

[74]  Christian Decker,et al.  Bitcoin meets strong consistency , 2014, ICDCN.

[75]  Yoad Lewenberg,et al.  SPECTRE: A Fast and Scalable Cryptocurrency Protocol , 2016, IACR Cryptol. ePrint Arch..

[76]  Elaine Shi,et al.  On Scaling Decentralized Blockchains - (A Position Paper) , 2016, Financial Cryptography Workshops.

[77]  Hubert Ritzdorf,et al.  On the Security and Performance of Proof of Work Blockchains , 2016, IACR Cryptol. ePrint Arch..

[78]  Prateek Saxena,et al.  A Secure Sharding Protocol For Open Blockchains , 2016, CCS.

[79]  Kartik Nayak,et al.  Solidus: An Incentive-compatible Cryptocurrency Based on Permissionless Byzantine Consensus , 2016, ArXiv.

[80]  Marko Vukolic,et al.  Eventually Returning to Strong Consistency , 2016, IEEE Data Eng. Bull..

[81]  David Wolinsky,et al.  Keeping Authorities "Honest or Bust" with Decentralized Witness Cosigning , 2015, 2016 IEEE Symposium on Security and Privacy (SP).

[82]  Elaine Shi,et al.  Snow White: Provably Secure Proofs of Stake , 2016, IACR Cryptol. ePrint Arch..

[83]  Emin Gün Sirer,et al.  Bitcoin-NG: A Scalable Blockchain Protocol , 2015, NSDI.

[84]  Elaine Shi,et al.  The Honey Badger of BFT Protocols , 2016, CCS.

[85]  Ariel Gabizon,et al.  Cryptocurrencies Without Proof of Work , 2014, Financial Cryptography Workshops.

[86]  Joseph Bonneau,et al.  Why Buy When You Can Rent? - Bribery Attacks on Bitcoin-Style Consensus , 2016, Financial Cryptography Workshops.

[87]  Emin Gün Sirer,et al.  Service-Oriented Sharding with Aspen , 2016, ArXiv.

[88]  Bryan Ford,et al.  Enhancing Bitcoin Security and Performance with Strong Consistency via Collective Signing , 2016, USENIX Security Symposium.

[89]  DR. Gavin Wood POLKADOT: VISION FOR A HETEROGENEOUS MULTI-CHAIN FRAMEWORK , 2016 .

[90]  Christian Cachin,et al.  Architecture of the Hyperledger Blockchain Fabric , 2016 .

[91]  Marko Vukolić,et al.  Rethinking Permissioned Blockchains , 2017 .

[92]  Elaine Shi,et al.  Hybrid Consensus: Efficient Consensus in the Permissionless Model , 2016, DISC.

[93]  Nikita Borisov,et al.  SmartCast: An Incentive Compatible Consensus Protocol Using Smart Contracts , 2017, Financial Cryptography Workshops.

[94]  Marko Vukolic,et al.  Blockchain Consensus Protocols in the Wild , 2017, DISC.

[95]  Carmela Troncoso,et al.  Systematizing Decentralization and Privacy: Lessons from 15 Years of Research and Deployments , 2017, Proc. Priv. Enhancing Technol..

[96]  Michael J. Fischer,et al.  Scalable Bias-Resistant Distributed Randomness , 2017, 2017 IEEE Symposium on Security and Privacy (SP).

[97]  George Danezis,et al.  The Road to Scalable Blockchain Designs , 2017, Login: The Usenix Magazine.

[98]  Alysson Neves Bessani,et al.  Elastic State Machine Replication , 2017, IEEE Transactions on Parallel and Distributed Systems.

[99]  Iddo Bentov,et al.  Tortoise and Hares Consensus: the Meshcash Framework for Incentive-Compatible, Scalable Cryptocurrencies , 2017, IACR Cryptol. ePrint Arch..

[100]  Eleftherios Kokoris Kogias,et al.  Poster : Scalable Bias-Resistant Distributed Randomness , 2017 .

[101]  Yacov Manevich,et al.  Scalable communication middleware for permissioned distributed ledgers , 2017, SYSTOR.

[102]  Georg Fuchsbauer,et al.  SpaceMint: A Cryptocurrency Based on Proofs of Space , 2018, ERCIM News.

[103]  Bryan Ford,et al.  OmniLedger: A Secure, Scale-Out, Decentralized Ledger , 2017, IACR Cryptol. ePrint Arch..

[104]  Silvio Micali,et al.  Algorand: Scaling Byzantine Agreements for Cryptocurrencies , 2017, IACR Cryptol. ePrint Arch..

[105]  Beng Chin Ooi,et al.  BLOCKBENCH: A Framework for Analyzing Private Blockchains , 2017, SIGMOD Conference.

[106]  Jason Teutsch,et al.  SmartPool: Practical Decentralized Pooled Mining , 2017, USENIX Security Symposium.

[107]  Fan Zhang,et al.  REM: Resource-Efficient Mining for Blockchains , 2017, IACR Cryptol. ePrint Arch..

[108]  Aggelos Kiayias,et al.  Ouroboros Praos: An adaptively-secure, semi-synchronous proof-of-stake protocol , 2017, IACR Cryptol. ePrint Arch..

[109]  D. He,et al.  Fintech and Financial Services , 2017 .

[110]  Bart Preneel,et al.  Publish or Perish: A Backward-Compatible Defense Against Selfish Mining in Bitcoin , 2017, CT-RSA.

[111]  Ghassan O. Karame,et al.  Towards Scalable and Private Industrial Blockchains , 2017, SEMA SIMAI Springer Series.

[112]  Jeremy Clark,et al.  Bitcoin's academic pedigree , 2017, ACM Queue.

[113]  Aviv Zohar Securing and scaling cryptocurrencies , 2017, IJCAI.

[114]  Christian Cachin,et al.  Blockchains and Consensus Protocols: Snake Oil Warning , 2017, 2017 13th European Dependable Computing Conference (EDCC).

[115]  Justin Cappos,et al.  CHAINIAC: Proactive Software-Update Transparency via Collectively Signed Skipchains and Verified Builds , 2017, USENIX Security Symposium.

[116]  Johan A. Pouwelse,et al.  Implicit Consensus: Blockchain with Unbounded Throughput , 2017, ArXiv.

[117]  Kartik Nayak,et al.  Practical Synchronous Byzantine Consensus , 2017, IACR Cryptol. ePrint Arch..

[118]  Edgar R. Weippl,et al.  Agreement with Satoshi - On the Formalization of Nakamoto Consensus , 2018, IACR Cryptol. ePrint Arch..

[119]  George Danezis,et al.  Chainspace: A Sharded Smart Contracts Platform , 2017, NDSS.

[120]  Ueli Maurer,et al.  But Why does it Work? A Rational Protocol Design Treatment of Bitcoin , 2018, IACR Cryptol. ePrint Arch..

[121]  Aggelos Kiayias,et al.  Ouroboros Praos: An Adaptively-Secure, Semi-synchronous Proof-of-Stake Blockchain , 2018, EUROCRYPT.

[122]  Haibin Zhang,et al.  BEAT: Asynchronous BFT Made Practical , 2018, CCS.

[123]  Aggelos Kiayias,et al.  SoK: A Consensus Taxonomy in the Blockchain Era , 2020, IACR Cryptol. ePrint Arch..

[124]  Snowflake to Avalanche : A Novel Metastable Consensus Protocol Family for Cryptocurrencies Team Rocket , 2018 .

[125]  Agenda ! Part I : Consensus Protocols " Traditional mechanisms " Blockchain consensus ! Part II : RapidChain [ CCS 2018 ] " Sharding-based consensus " Protocol overview " Results , 2018 .

[126]  Sarah Meiklejohn,et al.  Betting on Blockchain Consensus with Fantomette , 2018, ArXiv.

[127]  Mohammad Hossein Manshaei,et al.  A Game-Theoretic Analysis of Shard-Based Permissionless Blockchains , 2018, IEEE Access.

[128]  Philipp Jovanovic,et al.  OmniLedger: A Secure, Scale-Out, Decentralized Ledger via Sharding , 2018, 2018 IEEE Symposium on Security and Privacy (SP).

[129]  Pramod Viswanath,et al.  Compounding of Wealth in Proof-of-Stake Cryptocurrencies , 2018, Financial Cryptography.

[130]  Aggelos Kiayias,et al.  Reward Sharing Schemes for Stake Pools , 2018, 2020 IEEE European Symposium on Security and Privacy (EuroS&P).

[131]  George Danezis,et al.  Replay Attacks and Defenses Against Cross-shard Consensus in Sharded Distributed Ledgers , 2019, 2020 IEEE European Symposium on Security and Privacy (EuroS&P).