Building high throughput permissioned blockchain fabrics

Since the introduction of Bitcoin—the first widespread application driven by blockchains—the interest in the design of blockchain-based applications has increased tremendously. At the core of these applications are consensus protocols that securely replicate client requests among all replicas, even if some replicas are Byzantine faulty. Unfortunately, these consensus protocols typically have low throughput, and this lack of performance is often cited as the reason for the slow wider adoption of blockchain technology. Consequently, many works focus on designing more efficient consensus protocols to increase throughput of consensus. We believe that this focus on consensus protocols only explains part of the story. To investigate this belief, we raise a simple question: Can a well-crafted system using a classical consensus protocol outperform systems using modern protocols? In this tutorial, we answer this question by diving deep into the design of blockchain systems. Further, we take an in-depth look at the theory behind consensus, which can help users select the protocol that best-fits their requirements. Finally, we share our vision of high-throughput blockchain systems that operate at large scales. PVLDB Reference Format: Suyash Gupta, Jelle Hellings, Sajjad Rahnama and Mohammad Sadoghi. Building High Throughput Permissioned Blockchain Fabrics: Challenges and Opportunities. PVLDB, (): xxxx-yyyy, 2020. DOI:

[1]  Miguel Castro,et al.  Practical byzantine fault tolerance and proactive recovery , 2002, TOCS.

[2]  Suyash Gupta,et al.  Permissioned Blockchain Through the Looking Glass: Architectural and Implementation Lessons Learned , 2019, ArXiv.

[3]  E. Brewer,et al.  CAP twelve years later: How the "rules" have changed , 2012, Computer.

[4]  Jelle Hellings,et al.  Coordination-Free Byzantine Replication with Minimal Communication Costs , 2020, ICDT.

[5]  Vivien Quéma,et al.  RBFT: Redundant Byzantine Fault Tolerance , 2013, 2013 IEEE 33rd International Conference on Distributed Computing Systems.

[6]  Jelle Hellings,et al.  Blockchain consensus unraveled: virtues and limitations , 2020, DEBS.

[7]  Jean-Philippe Martin,et al.  Fast Byzantine Consensus , 2006, IEEE Transactions on Dependable and Secure Computing.

[8]  Shlomo Moran,et al.  Extended Impossibility Results for Asynchronous Complete Networks , 1987, Inf. Process. Lett..

[9]  Kartik Nayak,et al.  Synchronous Byzantine Agreement with Expected O(1) Rounds, Expected O(n2) Communication, and Optimal Resilience , 2019, IACR Cryptol. ePrint Arch..

[10]  Marko Vukolic,et al.  The Complexity of Asynchronous Byzantine Consensus , 2004 .

[11]  Yuanyuan Yang,et al.  A Survey of IoT Applications in Blockchain Systems , 2020, ACM Comput. Surv..

[12]  Suyash Gupta,et al.  Scaling Blockchain Databases through Parallel Resilient Consensus Paradigm , 2019, ArXiv.

[13]  Nancy A. Lynch,et al.  A Lower Bound for the Time to Assure Interactive Consistency , 1982, Inf. Process. Lett..

[14]  Carsten Binnig,et al.  BlockchainDB - A Shared Database on Blockchains , 2019, Proc. VLDB Endow..

[15]  Nancy A. Lynch,et al.  Brewer's conjecture and the feasibility of consistent, available, partition-tolerant web services , 2002, SIGA.

[16]  Leslie Lamport,et al.  Reaching Agreement in the Presence of Faults , 1980, JACM.

[17]  Satoshi Nakamoto Bitcoin : A Peer-to-Peer Electronic Cash System , 2009 .

[18]  Johannes Behl,et al.  CheapBFT: resource-efficient byzantine fault tolerance , 2012, EuroSys '12.

[19]  Jelle Hellings,et al.  Brief Announcement: The Fault-Tolerant Cluster-Sending Problem , 2019, DISC.

[20]  Bruce M. Kapron,et al.  Lower bound for scalable Byzantine Agreement , 2006, PODC '06.

[21]  Danny Dolev,et al.  Authenticated Algorithms for Byzantine Agreement , 1983, SIAM J. Comput..

[22]  Silvio Micali,et al.  Algorand: Scaling Byzantine Agreements for Cryptocurrencies , 2017, IACR Cryptol. ePrint Arch..

[23]  Gadi Taubenfeld,et al.  Possibility and impossibility results in a shared memory environment , 1989, Acta Informatica.

[24]  Patrick Valduriez,et al.  Principles of Distributed Database Systems , 1990 .

[25]  Nancy A. Lynch,et al.  Impossibility of distributed consensus with one faulty process , 1983, PODS '83.

[26]  Marko Vukolic,et al.  The Next 700 BFT Protocols , 2015, ACM Trans. Comput. Syst..

[27]  Carsten Binnig,et al.  BlockchainDB - Towards a Shared Database on Blockchains , 2019, SIGMOD Conference.

[28]  Danny Dolev,et al.  On the minimal synchronism needed for distributed consensus , 1983, 24th Annual Symposium on Foundations of Computer Science (sfcs 1983).

[29]  Michael Okun,et al.  On the round complexity of Byzantine agreement without initial set-up , 2009, Inf. Comput..

[30]  Kevin A Clauson,et al.  Geospatial blockchain: promises, challenges, and scenarios in health and healthcare , 2018, International Journal of Health Geographics.

[31]  Danny Dolev,et al.  Unanimity in an unknown and unreliable environment , 1981, 22nd Annual Symposium on Foundations of Computer Science (sfcs 1981).

[32]  Divyakant Agrawal,et al.  CAPER: A Cross-Application Permissioned Blockchain , 2019, Proc. VLDB Endow..

[33]  Eric A. Brewer,et al.  Towards robust distributed systems (abstract) , 2000, PODC '00.

[34]  Beng Chin Ooi,et al.  BLOCKBENCH: A Framework for Analyzing Private Blockchains , 2017, SIGMOD Conference.

[35]  Silvio Micali,et al.  Optimal algorithms for Byzantine agreement , 1988, STOC '88.

[36]  Jelle Hellings,et al.  ResilientDB: Global Scale Resilient Blockchain Fabric , 2020, Proc. VLDB Endow..

[37]  Nancy A. Lynch,et al.  Cryptographic protocols , 1982, STOC '82.

[38]  Michael Stonebraker,et al.  OLTP through the looking glass, and what we found there , 2008, SIGMOD Conference.

[39]  Ramakrishna Kotla,et al.  Zyzzyva , 2007, SOSP.

[40]  Danny Dolev,et al.  The Byzantine Generals Strike Again , 1981, J. Algorithms.

[41]  Marko Vukolic,et al.  Blockchain Consensus Protocols in the Wild (Keynote Talk) , 2017, DISC.

[42]  Alysson Neves Bessani,et al.  State Machine Replication for the Masses with BFT-SMART , 2014, 2014 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks.

[43]  Michael Stonebraker,et al.  An Evaluation of Distributed Concurrency Control , 2017, Proc. VLDB Endow..

[44]  Ittai Abraham,et al.  HotStuff: BFT Consensus with Linearity and Responsiveness , 2019, PODC.

[45]  Faisal Nawab,et al.  Blockplane: A Global-Scale Byzantizing Middleware , 2019, 2019 IEEE 35th International Conference on Data Engineering (ICDE).

[46]  Gang Chen,et al.  Untangling Blockchain: A Data Processing View of Blockchain Systems , 2017, IEEE Transactions on Knowledge and Data Engineering.

[47]  Senthil Nathan,et al.  Blockchain Meets Database: Design and Implementation of a Blockchain Relational Database , 2019, Proc. VLDB Endow..

[48]  Suyash Gupta,et al.  Proof-of-Execution: Reaching Consensus through Fault-Tolerant Speculation , 2019, EDBT.

[49]  Suyash Gupta,et al.  Brief Announcement: Revisiting Consensus Protocols through Wait-Free Parallelization , 2019, DISC.

[50]  Ee-Chien Chang,et al.  Towards Scaling Blockchain Systems via Sharding , 2018, SIGMOD Conference.

[51]  Jim Gray,et al.  Notes on Data Base Operating Systems , 1978, Advanced Course: Operating Systems.

[52]  Leslie Lamport,et al.  The Byzantine Generals Problem , 1982, TOPL.

[53]  Miguel Correia,et al.  Low complexity Byzantine-resilient consensus , 2005, Distributed Computing.

[54]  Marko Vukolic,et al.  Hyperledger fabric: a distributed operating system for permissioned blockchains , 2018, EuroSys.

[55]  Christian Catalini,et al.  Blockchain Technology for Healthcare: Facilitating the Transition to Patient-Driven Interoperability , 2018, Computational and structural biotechnology journal.

[56]  William E. Winkler,et al.  Data quality and record linkage techniques , 2007 .

[57]  Thomas Redman,et al.  The impact of poor data quality on the typical enterprise , 1998, CACM.

[58]  Daniel Davis Wood,et al.  ETHEREUM: A SECURE DECENTRALISED GENERALISED TRANSACTION LEDGER , 2014 .

[59]  Jeremy Clark,et al.  Bitcoin's academic pedigree , 2017, ACM Queue.

[60]  Jelle Hellings,et al.  An In-Depth Look of BFT Consensus in Blockchain: Challenges and Opportunities , 2019, Middleware Tutorials.

[61]  Leslie Lamport,et al.  The Implementation of Reliable Distributed Multiprocess Systems , 1978, Comput. Networks.

[62]  Matt Juden Blockchain and Economic Development : Hype vs . Reality , 2017 .

[63]  Dale Skeen,et al.  A Quorum-Based Commit Protocol , 1982, Berkeley Workshop.

[64]  Maurice Herlihy,et al.  Blockchains from a distributed computing perspective , 2019, Commun. ACM.

[65]  Nancy A. Lynch,et al.  Consensus in the presence of partial synchrony , 1988, JACM.