Building high throughput permissioned blockchain fabrics

Since the introduction of Bitcoin---the first widespread application driven by blockchains---the interest in the design of blockchain-based applications has increased tremendously. At the core of these applications are consensus protocols that securely replicate client requests among all replicas, even if some replicas are Byzantine faulty. Unfortunately, these consensus protocols typically have low throughput, and this lack of performance is often cited as the reason for the slow wider adoption of blockchain technology. Consequently, many works focus on designing more efficient consensus protocols to increase throughput of consensus. We believe that this focus on consensus protocols only explains part of the story. To investigate this belief, we raise a simple question: Can a well-crafted system using a classical consensus protocol outperform systems using modern protocols? In this tutorial, we answer this question by diving deep into the design of blockchain systems. Further, we take an in-depth look at the theory behind consensus, which can help users select the protocol that best-fits their requirements. Finally, we share our vision of high-throughput blockchain systems that operate at large scales.

[1]  Miguel Castro,et al.  Practical byzantine fault tolerance and proactive recovery , 2002, TOCS.

[2]  Suyash Gupta,et al.  Permissioned Blockchain Through the Looking Glass: Architectural and Implementation Lessons Learned , 2019, ArXiv.

[3]  Jean-Philippe Martin,et al.  Fast Byzantine Consensus , 2006, IEEE Transactions on Dependable and Secure Computing.

[4]  Shlomo Moran,et al.  Extended Impossibility Results for Asynchronous Complete Networks , 1987, Inf. Process. Lett..

[5]  Jelle Hellings,et al.  An In-Depth Look of BFT Consensus in Blockchain: Challenges and Opportunities , 2019, Middleware Tutorials.

[6]  Nancy A. Lynch,et al.  Cryptographic protocols , 1982, STOC '82.

[7]  Michael Stonebraker,et al.  OLTP through the looking glass, and what we found there , 2008, SIGMOD Conference.

[8]  Nancy A. Lynch,et al.  A Lower Bound for the Time to Assure Interactive Consistency , 1982, Inf. Process. Lett..

[9]  Marko Vukolic,et al.  The Next 700 BFT Protocols , 2015, ACM Trans. Comput. Syst..

[10]  Ittai Abraham,et al.  HotStuff: BFT Consensus with Linearity and Responsiveness , 2019, PODC.

[11]  Suyash Gupta,et al.  Proof-of-Execution: Reaching Consensus through Fault-Tolerant Speculation , 2019, EDBT.

[12]  Suyash Gupta,et al.  Brief Announcement: Revisiting Consensus Protocols through Wait-Free Parallelization , 2019, DISC.

[13]  Ee-Chien Chang,et al.  Towards Scaling Blockchain Systems via Sharding , 2018, SIGMOD Conference.

[14]  Jelle Hellings,et al.  Brief Announcement: The Fault-Tolerant Cluster-Sending Problem , 2019, DISC.

[15]  Danny Dolev,et al.  The Byzantine Generals Strike Again , 1981, J. Algorithms.

[16]  Ramakrishna Kotla,et al.  Zyzzyva , 2007, SOSP.

[17]  Danny Dolev,et al.  Unanimity in an unknown and unreliable environment , 1981, 22nd Annual Symposium on Foundations of Computer Science (sfcs 1981).

[18]  Divyakant Agrawal,et al.  CAPER: A Cross-Application Permissioned Blockchain , 2019, Proc. VLDB Endow..

[19]  Senthil Nathan,et al.  Blockchain Meets Database: Design and Implementation of a Blockchain Relational Database , 2019, Proc. VLDB Endow..

[20]  Leslie Lamport,et al.  Reaching Agreement in the Presence of Faults , 1980, JACM.

[21]  Suyash Gupta,et al.  Scaling Blockchain Databases through Parallel Resilient Consensus Paradigm , 2019, ArXiv.

[22]  Beng Chin Ooi,et al.  BLOCKBENCH: A Framework for Analyzing Private Blockchains , 2017, SIGMOD Conference.

[23]  Alysson Neves Bessani,et al.  State Machine Replication for the Masses with BFT-SMART , 2014, 2014 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks.

[24]  Danny Dolev,et al.  Authenticated Algorithms for Byzantine Agreement , 1983, SIAM J. Comput..

[25]  Nancy A. Lynch,et al.  Impossibility of distributed consensus with one faulty process , 1983, PODS '83.

[26]  Kartik Nayak,et al.  Synchronous Byzantine Agreement with Expected O(1) Rounds, Expected O(n2) Communication, and Optimal Resilience , 2019, IACR Cryptol. ePrint Arch..

[27]  Carsten Binnig,et al.  BlockchainDB - A Shared Database on Blockchains , 2019, Proc. VLDB Endow..

[28]  Nancy A. Lynch,et al.  Brewer's conjecture and the feasibility of consistent, available, partition-tolerant web services , 2002, SIGA.

[29]  Daniel Davis Wood,et al.  ETHEREUM: A SECURE DECENTRALISED GENERALISED TRANSACTION LEDGER , 2014 .

[30]  Jelle Hellings,et al.  ResilientDB: Global Scale Resilient Blockchain Fabric , 2020, Proc. VLDB Endow..

[31]  William E. Winkler,et al.  Data quality and record linkage techniques , 2007 .

[32]  Thomas Redman,et al.  The impact of poor data quality on the typical enterprise , 1998, CACM.

[33]  Satoshi Nakamoto Bitcoin : A Peer-to-Peer Electronic Cash System , 2009 .

[34]  E. Brewer,et al.  CAP twelve years later: How the "rules" have changed , 2012, Computer.

[35]  Jelle Hellings,et al.  Coordination-Free Byzantine Replication with Minimal Communication Costs , 2020, ICDT.

[36]  Vivien Quéma,et al.  RBFT: Redundant Byzantine Fault Tolerance , 2013, 2013 IEEE 33rd International Conference on Distributed Computing Systems.

[37]  Gadi Taubenfeld,et al.  Possibility and impossibility results in a shared memory environment , 1989, Acta Informatica.

[38]  Leslie Lamport,et al.  The Byzantine Generals Problem , 1982, TOPL.

[39]  Marko Vukolic,et al.  Hyperledger fabric: a distributed operating system for permissioned blockchains , 2018, EuroSys.

[40]  Faisal Nawab,et al.  Blockplane: A Global-Scale Byzantizing Middleware , 2019, 2019 IEEE 35th International Conference on Data Engineering (ICDE).

[41]  Michael Stonebraker,et al.  An Evaluation of Distributed Concurrency Control , 2017, Proc. VLDB Endow..